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Abstract 


This  study  compared  the  elements  of  two  existing  contracting  certification  programs 
to  an  ideal  contracting  certification  model  developed  by  a  panel  of  experts.  The  expert 
panel,  as  identified  by  the  research  team,  responded  to  an  open-ended  electronic 
interview  to  convey  their  ideas  regarding  the  individual  elements  for  the  model 
(experience  requirements,  education  requirements,  etc.).  Input  received  from  the  initial 
interviews  was  used  to  develop  a  survey  for  the  panel  members.  The  surveys  were 
completed  and  results  were  recorded  and  conveyed  back  to  the  panel  members.  This 
process  was  repeated  until  a  majority  consensus  was  reached  on  each  individual  element 
of  the  model,  resulting  in  the  ideal  certification  model.  The  final  model  contained 
elements  of  both  the  APDP  and  NCMA  certification  programs.  There  were  no  unique 
characteristics  identified  by  the  expert  panel.  Following  model  development,  stakeholder 
assessment  surveys  were  utilized  to  gain  insight  from  the  contracting  field  as  to  how  the 
ideal  model  was  perceived  when  compared  with  the  existing  certification  programs. 
Stakeholder  assessment  of  the  model  varied  according  to  each  individual  element 
contained  in  the  model. 

The  impact  of  this  study  was  aimed  at  ensuring  that  certified  contracting  professionals 
possess  the  levels  of  education,  experience,  and  contracting  knowledge  commensurate 
with  their  certification  level.  Potential  deficiencies  in  the  existing  certification  programs 
were  identified  and  recommendations  were  made  to  both  NCMA  and  senior  government 
contracting  personnel  for  potential  improvements  in  both  programs. 
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PROFESSIONAL  CONTRACTING  CERTIFICATION: 


AN  EXAMINATION  AND  MODEL  DEVELOPMENT 

I.  Introduction 

Background 

In  accordance  with  the  Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  of 
1990,  Air  Force  contracting  persoimel  are  currently  required  to  possess  a  level  of 
certification  commensurate  with  their  grade  and  position  in  order  to  continue  career 
progression  in  the  contracting  field.  Mandatory  certification  requirements  are  presently 
governed  by  the  Acquisition  Professional  Development  Program  (APDP),  as  instituted  by 
DAWIA.  This  three  tier  system  of  contracting  certification  requires  individuals  to 
possess  certain  levels  of  education,  job  experience,  and  various  Defense  Acquisition 
University  (DAU)  sponsored  contracting  courses. 

The  National  Contract  Management  Association  (NCMA)  also  sponsors  a  certification 
program.  Although  not  presently  required  for  government  personnel,  the  NCMA 
certification  program  offers  two  levels  of  contracting  certifications  in  addition  to  a 
Simplified  Acquisition  Specialist  credential.  The  requirements  for  certifications  are 
similar  to  the  government  sponsored  APDP  program  in  that  a  certain  level  of  education, 
job  experience,  and  at  least  two  contracting  courses  are  required  before  an  individual  can 
apply  for  certification.  In  addition,  perhaps  the  biggest  difference  between  NCMA  and 
APDP  certifications  is  that  NCMA  requires  the  successful  completion  of  a 
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comprehensive  examination  prior  to  granting  either  level  of  certification.  Furthermore, 
re-certification  is  mandatory  every  five  years  in  order  for  an  individual  to  maintain  the 
desired  level  of  certification. 

Problem  Statement 

The  problem  statement  of  this  thesis  proposal  recognizes  that  there  are  two 
certification  programs  in  existence  which  seek  to  accomplish  the  same  basic  objective — 
to  ensure  that  the  contracting  community  has  a  qualified,  knowledgeable,  and 
experienced  personnel  corps.  This  thesis  will  seek  to  construct  an  ideal  certification 
system  that  can  be  used  as  a  baseline  in  assessing  each  of  the  existing  systems.  The 
research  team  envisions  such  a  certification  system  to  make  clear  distinctions  between 
certification  levels  based  upon  education,  experience,  and  contracting  knowledge  as 
determined  by  highly  experienced  contracting  experts.  This  study  is  aimed  at  ensuring 
that  certified  contracting  professionals  possess  the  levels  of  education,  experience,  and 
contracting  knowledge  commensurate  with  their  certification  level. 

Research  Objectives 

This  research  had  three  primary  objectives: 

1 .  Identify  the  qualifications  necessary  for  government  contracting  personnel  to 
attain  professional  levels  of  certification. 

2.  Formulate  a  model  to  reflect  these  findings. 

3.  Compare  this  model  against  existing  certification  programs  to  identify  areas  in 
which  existing  certification  programs  can  be  improved. 
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Research  Questions 


In  accomplishing  the  above  stated  objectives,  the  research  team  intends  to  provide 
answers  to  the  following  questions: 

1 .  How  many  levels  of  certification  should  an  ideal  contracting  certification  program 
contain? 

2.  What  should  be  the  education  and  experience  requirements  for  contracting 
personnel  to  attain  certification? 

3.  Should  the  successfixl  completion  of  a  comprehensive  examination  be  required  for 
contracting  personnel  to  attain  certification? 

4.  Should  certified  contracting  personnel  be  subject  to  re-certification  throughout 
their  contracting  career? 

5.  What  aspects  of  the  ideal  certification  program  are  imique  to  existing  contracting 
certification  programs? 

By  identifying  any  potential  deficiencies  in  the  existing  certification  programs, 
recommendations  can  be  made  to  both  NCMA  and  senior  government  contracting 
personnel  for  improvement  in  either  or  both  contracting  certification  programs.  This  will 
potentially  lead  to  a  more  qualified  contracting  field  in  both  the  public  and  private  sector. 

Furthermore,  by  developing  an  ideal  contracting  certification  model  based  upon  the 
inputs  of  an  expert  panel,  the  importance  of  each  certification  element  (experience, 
formal  training,  education)  can  be  determined  and  then  compared  with  each  of  the 
existing  programs.  For  example,  the  ideal  certification  model  may  require  a  more 
stringent  education  requirement  but  a  relaxed  experience  requirement  for  each  level  of 
certification  when  compared  with  the  existing  programs.  In  this  event,  a  potential 
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discrepancy  could  be  identified  between  the  current  requirements  for  certification  and  the 
requirements  that  contracting  experts  feel  are  necessary  to  maintain  and  promote  a 
qualified  workforce.  As  a  result  of  this  study,  recommendations  can  be  made  to  the 
existing  contracting  certification  programs  to  potentially  enhance  the  certification  process 
and  ensure  a  qualified  contracting  workforce  in  both  the  public  and  private  sectors. 

The  results  of  this  study  were  expected  to  be  useful  in  identifying  the  qualifications 
that  contracting  personnel  should  possess  prior  to  being  granted  certification.  The  results 
may  support  present  contracting  certification  programs,  such  as  APDP  or  NCMA,  or  they 
may  recommend  some  potential  improvements  to  either  or  both  of  these  programs. 

Either  way,  by  gaining  insight  into  the  opinions  of  senior  contracting  experts,  this  study 
can  identify  the  qualifications  that  certified  contracting  personnel  should  exhibit. 

Definition  of  Terms 

The  following  key  terms  are  defined  as  they  relate  to  this  thesis  effort. 

1 .  Certification:  A  formal  declaration  that  a  person  possesses  the  required 
knowledge,  skills,  and  experience  to  be  recognized  as  proficient  in  their  field. 

2-  Contracting  Certification  Program:  A  system,  within  the  contracting 
profession,  comprised  of  various  levels  of  certification,  in  which  each  level 
encompasses  increased  requirements  on  knowledge,  skills,  and  experience. 

3.  Contracting  Officer:  An  individual  within  the  contracting  profession  with  the 
authority  to  enter  into,  administer,  and/or  terminate  contracts  and  make  related 
determinations  and  findings  (FAR,  1 998). 
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4.  Contracting  Personnel:  Personnel  whose  primary  occupation  is  that  of 
government  contracting  (i.e.,  1 102  GS  series,  64P  officer  AFSC).  These 
personnel  may  or  may  not  be  titled  as  “contracting  officer”. 

5.  Consensus:  A  measure  of  majority  agreement  on  survey  or  interview 
responses. 

6.  Delphi:  A  procedure  for  soliciting,  collating,  and  refining  expert  opinions  of  a 
group  to  arrive  at  an  accurate  group  response  (Brown,  1968). 

7.  Expert  (contracting):  A  senior  civilian  or  senior  officer  with  at  least  fifteen 
years  of  contracting  experience  and  is  either  presently,  or  has  in  the  past, 
supervised  contracting  personnel. 

8.  Qualifications:  The  prerequisites  one  must  hold  in  order  to  be  eligible  for 
certification.  Qualifications  can  be  in  terms  of  educational  requirements,  job 

-  experience,  knowledge  base,  or  other  factors  deemed  necessary  for 
certification. 

9.  Senior  Civilian:  An  employee  of  the  U.S.  Government  in  the  grade  of  GS-14 
or  higher,  including  the  Senior  Executive  Service  (SES). 

10.  Senior  Officer:  A  member  of  the  Armed  Forces  presently  serving  in  the  grade 
of  0-6  (colonel)  or  above. 

Scope 

Although  the  area  of  contracting  certification  applies  across  the  entire  Department  of 
Defense,  this  research  was  primarily  conducted  based  upon  the  expert  opinion  and 
experiences  of  Air  Force  personnel.  The  research  team  has  incorporated  experts  from  the 
Army  zind  Navy  into  the  expert  panel;  however,  the  representation  of  the  various  military 
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branches  is  by  no  means  proportionate  with  that  of  the  DoD-wide  contracting  career  field. 
The  time  constraints  of  this  research  effort  and  the  availability  of  cross-service  experts 
restricted  the  size  of  the  panel  and  the  diversity  of  contracting  backgrounds.  This 
research  is  qualitative  and  exploratory  in  nature  rather  than  quantitative.  For  this  reason, 
the  expert  panel  was  not  randomly  selected,  but  was  specifically  chosen  by  the  research 
team  to  obtain  the  opinions  of  the  most  knowledgeable  and  experienced  contracting 
experts  possible,  given  such  resource  constraints  as  research  time,  limited  funding,  travel 
restrictions,  and  the  availability  of  qualified  panel  participants.  The  research  team 
viewed  this  approach,  and  the  results  obtained  herein,  to  be  reasonable  and  valid. 

The  next  section  contains  the  literature  review.  It  discusses  the  importance  of 
professional  certification  as  well  as  existing  research  in  this  area  followed  by  a 
description  of  the  requirements  for  existing  contracting  certification  programs. 

Following  the  literature  review,  we  discuss  the  methodology  employed  to  implement  our 
research.  Specifically,  this  section  will  discuss  how  the  research  team  employed  the 
Delphi  technique  in  the  development  of  an  ideal  contracting  certification  model.  It  also 
reports  the  results,  as  determined  by  the  expert  panel,  of  the  formulation  for  the  ideal 
contracting  certification  model.  We  then  discuss  analysis  of  the  stakeholders  assessment 
and  conclude  with  a  summary  of  our  findings,  recommendations,  and  suggestions  for 
further  research. 
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II.  Literature  Review 


This  literature  review  will  identify  the  importance  of  professional  certification  in 
general,  identify  properties  and  characteristics  that  are  common  across  well  accepted 
professional  certification  programs,  examine  existing  professional  certification  programs 
in  contracting,  and  explain  the  exact  requirements  for  certification  in  the  Acquisition 
Professional  Development  Program  and  National  Contract  Management  Association. 
Following  a  review  of  contracting  related  certifications,  we  will  then  conduct  a  brief 
review  of  certification  programs  from  other  professions.  The  chapter  ends  with  a  brief 
comparison  of  the  existing  certification  programs  to  other  programs. 

Professional  Certification 

Certification  has  as  its  primary  goal  the  promotion  of  competencies  (Wiley,  1995).  It 
establishes  an  accepted  minimum  level  of  competence  within  a  profession  and  ensures 
that  persons  in  a  particular  field  are  adequately  qualified  to  practice  a  profession.  A  study 
conducted  in  1980  reported  that  professional  competencies  improved  in  54%  of  the 
organizations  studied  as  a  direct  result  of  their  certification  efforts  (Bratton  and 
Hildebrand,  1980).  Certification  programs  allow  practitioners  in  fields  such  as 
accounting,  medicine,  law,  and  contracting  to  demonstrate  a  particular  level  of 
knowledge  and/or  experience  in  a  chosen  field  (Wiley,  1995).  An  effective  certification 
program  can  inform  both  employers  and  applicants  about  what  training  and  skills  are 
most  fundamental  to  good  practice,  and  it  can  differentiate  these  from  related  but 
secondary  credentials  such  as  personal  traits  and  characteristics  which  may  be  beneficial 
to  a  profession  (Imrey,  1994). 


7 


Another  goal  of  certification  programs  is  the  overall  improvement  of  the  profession. 
Achievement  of  a  standardized  level  of  certification  ensures  a  profession  that  its  members 
are  competent.  “Certification  requires  that  the  sponsoring  association  be  postured  to  limit 
incompetent  practitioners  or  to  improve  performance  by  establishing  competency 
standards”  (Wiley,  1995:  272).  More  importantly,  certification  is  tangible  evidence  that  a 
person  has  met  minimal  performance  standards,  possesses  requisite  education  and 
experience,  and  has  demonstrated  the  skills  to  perform  in  the  profession  (Wiley,  1995). 
Certification  provides  evidence  to  others  that  a  professional  standard  has  been 
established. 

Many  established  professions  have  developed  certification  programs  to  ensure  the 
integrity  of  their  professions.  Professional  associations  administer  the  programs  that 
typically  consist  of  a  combination  of  educational  and  experience  requirements  as  well  as 
the  successful  completion  of  a  comprehensive  examination  or  series  of  examinations. 

After  initial  certification,  many  programs  possess  continuing  education  requirements  to 
ensure  that  an  individual  remains  current  in  his  or  her  field  of  expertise. 

As  previously  stated,  the  first  goal  in  the  development  of  an  effective  certification 
program  involves  establishing  an  accepted  minimum  level  of  competence  within  a 
profession.  A  true  profession  must  legitimate  not  only  what  it  does,  but  also  how  it  is 
done  (Abbott,  1988: 189).  An  effective  certification  program  will  ensure  those  outside 
the  profession  that  work  performed  in  a  profession  is  done  in  a  competent,  efficient, 
manner. 

Certification  at  the  individual  level  is  a  process  by  which  individuals  can  achieve 
professional  recognition  for  professional  knowledge  and  achievement,  whether  or  not 
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accompanied  by  a  degree  (Imrey,  1994).  Individual  benefits  of  certification  include  a 
mastery  of  the  profession’s  body  of  knowledge,  public  recognition,  currency,  career 
advancement,  pay  incentives,  and  a  professional  attitude  (Wiley,  1995).  The  most 
important  long-run  benefits  of  certification  may  be  attained  through  any  voluntary 
program  that  is  technically  sound,  fairly  administered,  recognizes  the  breadth  and 
diversity  of  a  profession,  and  asserts  the  profession’s  commitments  to  quality  (Imrey, 
1994). 

Examples  of  Professional  Certification  Programs 

Three  well-known  certification  programs  are  those  for  public  accounting,  medical 
licensing,  and  the  legal  profession.  In  this  next  section  we  review  the  requirements  of 
these  three  systems  to  gain  a  better  understanding  of  how  others  have  attempts  to  meet 
the  goals  of  certification  programs.  We  also  review  the  certification  programs  of  a 
similar  field  to  contracting  -  the  National  Association  of  Purchasing. 

The  recognized  standard  in  the  public  accounting  profession  is  the  Certified  Public 
Accountant  (CPA)  certification.  In  order  to  legally  practice  public  accounting, 
individuals  must  possess  a  license.  To  become  licensed,  they  must  first  possess  a  CPA 
certificate.  All  candidates  for  the  CPA  certification  must  pass  the  Uniform  CPA 
Examination  that  is  administered  by  the  American  Institute  of  Certified  Public 
Accountants.  In  addition,  each  state  has  its  own  set  of  education,  experience,  and  other 
requirements  that  individuals  must  meet. 

Education  requirements  of  most  states  mandate  at  least  a  bachelor’s  degree  in 
accounting  with  some  requiring  a  minimum  of  150  hours  of  study.  A  minority  of  states 
require  individuals  to  possess  a  master’s  degree. 
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Experience  requirements  vary  'widely.  Most  states  require  at  least  two  years  public 
accounting  experience.  Some  accept  non-public  accounting  experience.  However,  these 
states  typically  require  more  years  of  experience. 

Some  states  have  implemented  a  tiered  system  for  certification  and  licensing.  States 
with  a  one-tiered  system  require  candidates  to  pass  the  CPA  examination  and  fulfill  the 
experience  requirements  to  obtain  both  the  certificate  and  license.  Two-tiered  systems 
grant  the  certification  upon  completion  of  the  exam,  but  'withhold  licenses  until 
experience  requirements  have  been  met. 

In  conjunction  with  education  and  experience  requirements,  states  issue  general 
qualifications  such  as  age,  citizenship,  and  residency.  The  most  common  age 
requirement  is  that  candidates  be  1 8  years  of  age.  However,  a  few  states  require 
candidates  to  be  at  least  21  years  of  age  and  some  states  have  no  age  requirements  at  all. 
The  vast  majority  of  states  do  not  require  citizenship  but  do  require  residency  in  that 
respective  state  (AICPA  homepage,  1 1  Feb  99). 

Similar  to  public  accounting,  medical  licensing  requirements  vary  according  to  state. 
The  Federation  of  State  Medical  Boards  and  the  National  Board  of  Medical  Examiners 
jointly  sponsor  the  United  States  Medical  Licensing  Examination  (USMLE)  for  medical 
licensing.  In  order  to  take  the  USMLE,  applicants  must  have  graduated  from  an 
accredited  school  of  medicine.  The  USMLE  is  a  three-step  examination,  each  of  which 
must  be  completed  in  sequence.  Upon  successful  completion  of  this  examination, 
applicants  must  apply  to  a  state’s  board  of  medical  licensing  to  receive  a  medical  license. 

Certification  to  practice  law  also  varies  by  state.  Each  state  requires  that  individuals 
pass  a  state’s  bar  examination  that  is  composed  of  a  multiple-choice  multi-state  section 
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and  an  essay  section  tailored  to  the  state  in  which  it  is  taken.  A  prerequisite  for  this  exam 
is  the  education  requirement  that  applicants  possess  a  juris  doctorate  degree  from  an 
accredited  school  of  law.  Successful  completion  of  the  examination  will  allow 
individuals  to  apply  to  a  state’s  board  of  bar  examiners  to  obtain  a  license  to  practice  in 
that  state.  Each  state’s  board  of  bar  examiners  has  various  other  requirements  that  will  be 
considered  in  determining  whether  or  not  to  issue  a  license.  These  requirements  range 
from  character  witness  statements  to  Federal  Bureau  of  Investigation  background  checks 
and  fingerprinting. 

To  maintain  a  license  to  practice  law,  attorneys  are  required  to  earn  continuing  legal 
education  credits  over  an  expressed  period  of  time  as  determined  by  each  state.  In 
general,  the  requirements  range  from  ten  to  sixteen  hours  annually  with  at  least  three 
hours  in  ethics-related  training.  The  research  team  found  no  evidence  of  experience 
requirements  prior  to  licensing. 

It  is  evident  from  the  research  team’s  review  of  certification  programs  that  a  number 
of  general  characteristics  are  common  to  all  programs.  Certification  programs  require 
some  level  of  fundamental  education,  related  experience,  and  demonstrated  success  on  a 
comprehensive  examination.  We  will  see  later  in  this  section  that  both  the  APDP  and 
NCMA  contracting  certification  programs  have  many  of  these  same  elements. 

Professional  Purchasing  Certifications 

This  section  discusses  the  existing  certification  programs  available  in  the  field  of 
purchasing.  It  is  important  to  note  that  while  certain  aspects  of  purchasing  and 
contracting  are  the  same,  the  certification  programs  discussed  below  are  not  substitutes 
for  the  previously  discussed  contracting  certification  programs  of  NCMA  and  APDP. 
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Purchasing  can  be  considered  a  subset  of  contracting  in  that  while  purchasing  managers 
possess  expertise  in  areas  such  as  supply  chain  management,  contracting  personnel  may 
be  employed  in  more  diverse  arenas.  For  example,  a  contracting  officer  may  be  buying 
office  furniture,  purchasing  janitorial  services,  or  procuring  a  new  weapon  system.  The 
following  information  on  purchasing  certifications  provides  a  point  of  reference  and 
facilitates  comparison  with  the  previously  mentioned  contracting  certifications. 

The  National  Association  of  Purchasing  Managers  is  a  professional  organization 
intended  to  provide  national  and  international  leadership  in  purchasing  as  well  as  supply 
chain  management  and  materials  management.  This  organization  originated  the  Certified 
Purchasing  Manager  (CPM)  certification  in  1974.  Applicants  for  the  CPM  certification 
must  successfully  complete  a  comprehensive  examination.  In  addition,  they  must  either 
have  five  years  of  full-time  professional  purchasing  and  supply  management  experience, 
or  have  a  four-year  degree  from  an  accredited  institution  and  three  years  of  full-time 
professional  purchasing  and  supply  management  experience.  Individuals  who  have 
earned  the  CPM  designation  are  required  to  be  recertified  every  five  years.  Re¬ 
certification  is  accomplished  according  to  the  organization’s  point  scale.  To  be  re¬ 
certified,  an  applicant  must  earn  twelve  CPM  points  during  their  current  certificate 
period.  A  minimum  of  eight  points  must  be  educational  in  nature  and  the  other  four  may 
be  earned  in  a  professional  contributions  category.  Categories  considered  educational  in 
nature  are  college  courses  either  taken  or  taught,  continuing  education  classes  either 
taken  or  taught,  and  the  CPM  examination  (NAPM  homepage,  25  Nov  98). 

Another  purchasing  certification  is  offered  by  the  National  Institute  of  Governmental 
Purchasing  (NIGP).  Like  the  National  Association  of  Purchasing  Managers,  NIGP  is 
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another  professional  organization  with  the  purpose  of  providing  education  and  technical 
assistance  for  the  government  purchasing  community.  This  organization  provides  two 
certifications,  the  Certified  Public  Purchasing  Officer  (CPPO)  certification  and  the 
Certified  Professional  Public  Buyer  (CPPB)  certification.  The  CPPB  designation  is 
designed  to  be  an  entry  level  certification,  whereas  the  CPPO  designation  denotes  a  more 
advanced  appointment  (Similar  to  the  CACM  and  CPCM  designations  under  the  NCMA 
program).  Requirements  to  sit  for  the  examinations  for  the  two  purchasing  certifications 
are  summarized  in  Tables  4  and  5  (NIGP  homepage,  1 1  Feb  99). 


Table  1.  CPPB  Examination  Eligibility  Requirements 


Schedule  A 

High  School  Diploma  or  GED 

6  college  credit  hours  in  purchasing  courses  or  48  contact  hours  in 
purchasing  seminars 

4  years  total  purchasing  experience  which  must  include  2  years  of 
current  public  purchasing  experience 

Schedule  B 

Associate's  degree 

6  college  credit  hours  in  purchasing  courses  or  48  contact  hours  in 
purchasing  seminars 

3  years  total  purchasing  experience  which  must  include  2  years  of 
current  public  purchasing  experience 

Schedule  C 

Bachelor’s  degree 

6  college  credit  hours  in  purchasing  courses  or  48  contact  hours  in 
purchasing  seminars 

2  years  of  current  public  purchasing  experience 

Schedule  D 

Advanced  degree  (Master's  or  Ph.D.) 

3  college  credit  hours  in  purchasing  courses  or  24  contact  hours  in 
purchasing  seminars 

2  years  of  current  public  purchasing  experience 
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Table  2.  CPPO  Examination  Eligibility  Requirements 


Schedule  A 

High  School  Diploma  or  GED 

6  college  credit  hours  in  purchasing  courses  or  48  contact  hours  in 
purchasing  seminars 

5  years  total  purchasing  experience:  4  years  current  public  purchasing 
experience,  of  which  2  years  are  in  a  public  purchasing  management 
function. 

Schedule  B 

Associate  degree 

6  college  credit  hours  in  purchasing  courses  or  48  contact  hours  in 
purchasing  seminars 

4  years  total  purchasing  experience:  3  years  current  public  purchasing 
experience,  of  which  2  years  are  in  a  public  purchasing  management 
function. 

Schedule  C 

Bachelor's  degree 

6  college  credit  hours  in  purchasing  courses  or  48  contact  hours  in 
purchasing  seminars 

3  years  total  purchasing  experience:  2  years  current  public  purchasing 
experience,  of  which  2  years  are  in  a  public  purchasing  management 
function. 

Schedule  D 

Advanced  degree  (Master's  or  Ph.D.) 

3  college  credit  hours  in  purchasing  courses  or  24  contact  hours  in 
purchasing  seminars 

3  years  total  purchasing  experience:  2  years  current  public  purchasing 
experience,  of  which  2  years  are  in  a  public  purchasing  management 
flmction. 

In  addition,  recertification  is  required  every  five  years  in  order  to  maintain  either  of 
these  designations.  Recertification  is  based  on  a  point  system  comprised  of  organization 
membership,  professional  contributions,  as  well  as  education  and  training. 


Professional  Contracting  Certifications 

As  previously  stated,  an  effective  certification  program  can  inform  both  employers 
and  applicants  about  what  training  and  skills  are  most  fundamental  to  good  practice,  and 
it  can  differentiate  these  from  related  but  secondary  credentials.  A  recent  study  has 
identified  knowledge,  skills,  and  abilities  critical  to  contracting  officers  as  identified  by 
contracting  managers  (Joyner,  1998).  The  study  identified  these  attributes  via 
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questionnaires  to  the  supervisors  of  contracting  officers.  Results  of  these  questionnaires 
are  presented  in  Table  3. 


Table  3.  Essential  Knowledge,  Skills,  Abilities,  and  Other  Characteristics  (Joyner  and 

Ucciardi,  1998) 


Knowledge 

Skills 

Abilities 

Other  Characteristics 

Regulations 

Negotiation 

Communication 

Ethical 

Business  Practices 

Computer  Literate 

Analytical 

Confident 

Technical 

Organized 

Interpersonal 

Accountable 

Law 

Business  Acumen 

Leadership 

Even-Tempered 

Bachelor’s  Degree 

Planning 

Flexibility 

Customer  Oriented 

Based  upon  the  desired  attributes  of  contracting  officers,  as  identified  by  Joyner  and 
Ucciardi,  it  follows  that  an  effective  certification  model  for  contracting  may  need  to  be 
constructed  aroimd  this  framework.  The  majority  of  these  attributes  lend  themselves  to 
being  learned  while  others  do  not.  Those  that  can  be  learned  and  measured  would  likely 
need  to  be  evaluated  in  a  certification  model.  It  is  realistic  that  through  a  combination  of 
job  experience,  education,  and  formal  training,  an  individual  has  the  potential  to  develop 
these  attributes.  While  it  can  be  argued  for  example  that  neither  ethics  nor  leadership  can 
be  taught  from  a  textbook,  contracting  experience  can  at  least  introduce  an  individual  to 
these  characteristics  and  continually  reinforce  their  importance  on  the  job.  It  is 
understandable  that  these  desired  attributes  should  be  captured  in  a  contracting 
certification  program  and  supports  the  requirements  for  experience,  formal  training,  and 
education  currently  incorporated  in  the  NCMA  and  APDP  programs. 
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The  contracting  profession  has  two  established  certification  programs,  one  for  DoD 
contracting  personnel  and  a  second  for  the  contracting  profession  in  general.  The  first 
certification  program  to  be  discussed  is  the  APDP  program.  In  an  attempt  to  improve 
Department  of  Defense  (DoD)  acquisition  training  and  in  response  to  a  number  of  highly 
publicized  procurement  blunders,  Congress  enacted  and  implemented  the  Defense 
Acquisition  Workforce  Improvement  Act  in  September  1990.  The  intent  of  the  Defense 
Acquisition  Workforce  Improvement  Act  was  to  develop  a  better-trained  and  more 
professional  DoD  workforce  and  to  reestablish  the  public’s  trust  in  government 
acquisition  personnel.  The  Defense  Acquisition  Workforce  Improvement  Act  resulted  in 
the  implementation  of  a  certified  training  curriculum  mandatory  for  progression  in  the 
contracting  career  field,  as  well  as  other  acquisition  specialties. 

The  three-tiered  APDP  system  mandates  certain  Defense  Acquisition  University 
courses  in  such  areas  as  contract  pricing,  contract  law,  and  contract  administration  for 
certification  in  the  contracting  field  (Office  of  the  Under  Secretary  of  Defense  for 
Acquisition  and  Technology!  996).  In  addition,  strict  experience  timelines  must  be  met 
before  an  individual  can  be  granted  certification  and  receive  a  contracting  warrant. 
Currently,  there  are  no  recertification  requirements.  Table  2  identifies  the  minimum 
requirements  for  each  certification  level  under  the  APDP  system  (OSDA&T,  1996: 
Appendix  F). 
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Table  4.  APDP  Certification  Requirements  (OUSDA&T,  1996) 


Certification  Level 

Mandatory  Education 

Minimum  Experience 

Entry  Level 

(I) 

Bachelors  Degree  w/24  business  related 
semester  hours  OR  10  years  experience 
Contracting  Fimdamentals 

Contract  Pricing 

1  Year 

Intermediate  Level 
(11) 

Intermediate  Contracting 

Intermediate  Contract  Pricing 
Government  Contract  Law 

2  Years 

Advanced  Level 
(III) 

Executive  Contracting 

Management  for  Contract  Supervisors 

4  Years 

For  an  individual  to  be  appointed  a  warranted  contracting  officer,  he  or  she  must 
possess  level  I  certification.  A  warranted  contracting  officer  is  the  only  person  who  may 
legally  obligate  government  funds.  However,  it  is  common  practice  in  the  career  field  to 
require  level  II  certification  before  making  such  an  appointment.  The  government  is 
somewhat  unique  in  this  respect.  Most  other  professions  encourage  certification  but 
allow  its  practice  without  it  (Wiley,  1995).  Certification  should  not  be  confused  with 
licensure  where  an  individual  is  prohibited  fi-om  practicing  a  profession  without  a  license. 

The  second  established  contracting  certification  program  is  sponsored  by  NCMA.  It 
consists  of  two  levels:  the  Certified  Associate  Contract  Manager  (CACM)  and  the 
Certified  Professional  Contract  Manager  (CPCM).  The  CACM  is  NCMA’s  entry-level 
certification  designation  while  the  CPCM  is  a  more  advanced  certification  than  the 
CACM.  According  to  NCMA,  the  purpose  of  its  certification  program  is  to  certify 
professional  growth  and  accomplishments,  and  to  improve  professionalism  (NCMA 
homepage,  1  Feb  99).  There  are  currently  no  requirements  for  NCMA  certification  to  be 
appointed  a  warranted  contracting  officer  for  the  government.  NCMA  encourages 
certification  for  a  number  of  reasons.  These  include  greater  respect  and  recognition, 
increased  competitiveness  in  the  job  market,  an  indication  of  personal  initiative,  a 
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validation  of  contracting  proficiency,  increases  professional  credibility,  demonstrates 
achievement,  knowledge,  and  skills  and  is  an  advantage  when  being  considered  for  hiring 
and  promotions  (NCMA  homepage). 

The  CACM  designation  recognizes  the  mastery  of  the  fundamentals  of  the 
government  contracting  profession  through  a  combination  of  formal  education, 
acquisition  education  and  training,  contracting  experience,  and  a  comprehensive 
examination.  The  CPCM  designation  recognizes  individuals  who  have  attained  a  high 
level  of  education,  experience,  and  training  in  the  procurement  and  contracting  profession 
through  an  even  more  stringent  set  of  requirements.  Requirements  for  the  respective 
certification  designations  are  presented  in  Table  3  (NCMA  homepage,  1  Feb  99). 

Table  5.  NCMA  Certification  Requirements  (NCMA  homepage) 
Certification  Requirements 

Entry  Level  -  1  year  college,  1  acquisition  course,  4  additional 

(CACM)  courses/years  fi-om  one  or  more  of  the  previous  three  categories 

“  1  year  contracting  experience 
--  Successful  completion  of  the  CACM  examination 
Advanced  Level  --  Bachelor’s  degree,  8  procurement  related  courses 

(CPCM)  —  2  years  contracting  experience 

_ —  Successful  completion  of  the  CPCM  examination 

Comparison  of  Certification  Programs 

The  research  team’s  review  of  certification  programs  reveals  that  many  characteristics 
are  common  to  all  programs.  All  of  the  certification  programs  reviewed  require  some 
level  of  fundamental  education,  related  experience,  and  demonstrated  success  on  a 
comprehensive  examination.  Both  the  APDP  and  NCMA  contracting  certification 
programs  have  many  of  these  same  elements  in  their  attempts  to  ensure  a  qualified 
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professional  workforce.  However,  of  the  programs  reviewed,  only  the  APDP  program 
does  not  require  the  comprehensive  examination. 

Based  on  this  review  of  other  effective  certification  programs,  we  will  develop  a 
preliminary  model  of  the  characterizations  of  an  ideal  certification  program  as  it  relates  to 
the  field  of  contracting.  We  intend  to  employ  a  modified  Delphi  technique  using  subject 
matter  experts  to  assist  with  the  development  of  an  ideal  model. 
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III.  Methodology 


Overview 

Based  on  the  review  of  other  effective  certification  programs,  the  research  team 
developed  a  preliminary  model  of  the  characterizations  of  an  ideal  certification  program 
as  it  relates  to  the  field  of  government  contracting.  To  satisfy  the  objectives  and  research 
questions  identified  in  Chapter  I,  the  research  team  employed  a  modified  Delphi 
technique  to  obtain  purposive  judgment  samples  from  a  variety  of  experts  in  the 
contracting  field. 

F ollowing  the  formulation  of  our  certification  model,  comparisons  were  made  with  the 
existing  certification  programs  (NCMA  and  APDP).  In  addition,  the  research  team 
conducted  a  widespread  assessment  of  the  model  by  soliciting  the  opinion  of  contracting 
certification  stakeholders.  Basically,  this  widespread  assessment  revealed  how 
contracting  personnel  might  react  to  the  implementation  of  this  new  system. 

Phases  of  Research.  The  following  four  phases  of  research  were  necessary  to 
achieve  our  research  objectives: 

1.  Basic  concepts  and  model  inputs  for  the  ideal  contracting  certification  program 
were  obtained  using  a  set  of  open-ended  written  interview  style  questions,  which  were 
electronically  mailed  to  each  member  of  the  expert  panel,  with  responses  being 
independently  evaluated  by  each  member  of  the  research  team. 

2.  A  questionnaire,  with  more  specific  questions  than  in  the  previous  phase,  was 
developed  based  upon  the  results  of  phase  1.  A  modified  Delphi  technique  was  applied 
to  this  questionnaire,  to  provide  a  set  of  detailed  inputs  for  the  model.  It  was  expected 


20 


that  at  least  two  rounds  of  feedback  and  adjustment  would  be  used  to  converge  on  a 
majority  agreement. 

3.  A  certification  model  was  formed  based  upon  the  consensus  formed  in  the 
previous  phase.  The  research  team  then  assessed  differences  between  this  model  and 
existing  certification  programs. 

4.  The  details  of  the  newly  developed  contracting  certification  program,  along 
with  a  questionnaire  was  sent  to  100  stakeholders  in  the  contracting  career  field  to  gain 
perceptions  on  whether  or  not  the  new  model  is  superior  to  those  already  in  existence. 

The  remainder  of  this  chapter  addresses  the  justification  for  the  selected 
methodology,  the  selection  and  assignment  of  experts,  the  data  analysis  and  decision 
criteria,  the  development  of  survey  instruments,  and  model  development. 

Justification  for  the  Methodology 

This  research  was  qualitative.  The  subject  matter  had  no  numerical  values  by  which 
measurement  was  possible.  Persoimel  competencies  and  job-related  qualifications  thus 
became  dependent  on  judgment.  In  such  cases,  the  best  possible  judgment  should  be 
obtained;  hence,  expert  opinion  becomes  valuable.  Decision  theory  holds  that,  under 
uncertainty,  the  quality  of  the  decision,  or  the  probability  of  a  correct  decision,  improves 
as  the  amount  of  information  increases  (Anderson,  1985). 

The  Delphi  technique  involves  the  use  of  subject  matter  experts  to  attain  an  ideal 
model.  The  Delphi  procedure  grew  out  of  experiments  from  the  RAND  Corporation  in 
the  late  1940’s  in  an  effort  to  enhance  the  reliability  of  forecasting.  Several  types  of 
group  response  techniques  were  attempted,  based  on  the  theory  that  “N  heads  are  better 
than  one.”  The  studies  at  RAND  indicated  three  main  disadvantages  of  using  group 
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discussion  and  committee  efforts  to  reach  accurate  group  responses.  These  disadvantages 
included  1)  influence  by  dominant  members,  2)  excess  noise  (extraneous  information), 
and  3)  group  pressure  for  compromise  and  conformity.  The  Delphi  technique  controls  for 
these  disadvantages  of  group  consensus  by  offering  anonymity,  controlled  feedback,  and 
statistical  group  response  (Nancarrow,  1987). 

There  are  several  properties  of  the  Delphi  technique  that  should  be  pointed  out.  The 
procedure  is,  above  all,  a  rapid  and  relatively  efficient  way  to  gain  the  opinions  of  a 
highly  select  group  of  experts.  The  feedback,  if  the  group  of  experts  involved  is  mutually 
self-respecting,  can  be  novel  and  interesting  to  all.  The  use  of  controlled  feedback  by  the 
research  team  lended  an  air  of  objectivity  to  the  outcomes  that  may  or  may  not  have  been 
spurious,  but  which  were  at  least  reassuring.  And  finally,  anonymity  and  group  response 
allowed  a  sharing  of  responsibility  and  a  release  from  the  respondents’  inhibitions.  The 
Delphi  technique  is  subject  to  greater  acceptance  on  the  part  of  the  group  than  are  the 
consensus’s  arrived  at  by  more  direct  forms  of  group  interaction  (Dalkey,  1969). 

This  research  effort  departs  from  the  traditional  Delphi  approach,  in  that  the  research 
team  attempted  to  streamline  the  number  of  iterations  by  dropping  off  solely  unique  or 
isolated  responses  during  the  interview  process.  For  example,  responses  provided  during 
the  interview  process  that  were  unique  to  a  single  user,  were  dismissed  and  not  offered  as 
responses  during  subsequent  rounds  of  questioning.  The  research  team  realizes  that  this 
approach  deviates  from  the  true  Delphi  technique  in  that  minority  opinions  are  quickly 
dismissed  and  not  further  offered  to  the  panel  for  consideration.  This  deviation  is 
incorporated  based  upon  the  limited  time  available  to  conduct  this  study  as  well  as  the 
time  considerations  of  the  expert  panel.  The  research  team  felt  that  excessive  iterations 
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of  similar  questionnaires  would  reduce  interest  and  participation  from  the  expert  panel, 
and  would  subsequently  detract  from  the  formulation  of  the  ideal  model.  By  eliminating 
isolated  responses  during  the  interview  process,  the  research  team  was  able  to  arrive  at  a 
majority  agreement  quickly  in  the  model  development  process,  while  retaining  maximum 
panel  member  participation. 

In  addition,  the  research  team  was  fully  aware  of  the  vulnerabilities  that  bias  can  have 
on  this  form  of  research.  Bias  can  occur  in  the  selection  of  experts,  topics  and  questions, 
the  wording  of  questions,  the  manner  of  interviewing,  and  the  interpretation  of  responses. 
(Shane,  1998).  The  research  team  made  every  effort  to  detect  and  minimize  bias.  One 
such  technique  used  was  that  the  research  team  independently  evaluated  the  questionnaire 
responses  and  then  reported  their  findings  to  one  another. 

Experts 

To  begin  collecting  data  for  the  ideal  contracting  certification  model,  it  was  first 
important  to  identify  the  population  from  which  to  select  a  representative  sample  of 
experts.  One  of  the  criticisms  of  the  Delphi  technique  is  that  literature  is  inconclusive 
concerning  the  value  of  experts  in  reaching  accurate  group  predictions  (Sackman,  1974). 
However,  for  the  purposes  of  basic  information  gathering,  the  use  of  experts  is 
considered  important  and  does  not  degrade  the  results  generated  by  using  the  Delphi 
technique  (Brown  &  Helmer,  1964).  Since  this  study  focused  on  a  contracting 
certification  model  for  government  contracting,  that  was  the  population  targeted. 

The  ideal  expert  panel  for  this  research  effort  should  be  representative  of  the  entire 
Department  of  Defense  contracting  corps,  since  this  research  was  focused  on  a 
certification  model  applicable  to  this  population.  For  a  sample  to  be  representative  of  this 
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population,  a  mix  of  military  officials  was  required  from  each  military  service,  as  well  as 
a  proportionate  number  of  civilians  from  each  service.  The  research  team  concedes  that  a 
sample  of  this  type  would  be  ideal  to  represent  the  Department  of  Defense  contracting 
corps.  However,  it  was  necessary  that  appropriate  trade-offs  were  made  in  this  area  due 
to  limited  contacts  and  interactions  within  the  various  military  branches.  The  research 
team  attempted  to  gain  a  diverse  mix  of  experts  for  the  panel,  but  for  the  purposes  of  time 
and  expedited  communications,  the  expert  panel  consisted  primarily  of  senior  military 
and  civilian  personnel  from  the  Air  Force  contracting  career  field. 

As  a  result,  the  research  team  envisioned  an  expert  panel  comprised  of  five  Air  Force 
colonels,  and  ten  civilians,  with  a  minimum  pay-grade  of  GS-14.  The  requirements  to  be 
considered  an  expert  and  serve  on  this  panel  include  1)  a  minimum  pay-grade  of  0-6  or 
GS-14, 2)  a  minimum  of  fifteen  years  contracting  experience,  and  3)  serving,  either 
presently  or  in  the  past,  as  a  supervisor  of  contracting  personnel.  The  research  team  feels 
that  these  prerequisites  provided  a  panel  with  considerable  knowledge  and  experience  in 
the  contracting  profession. 

The  selection  of  the  expert  panel  was  based  upon  the  above  criteria,  in  addition  to  the 
feasibility  or  realism  that  the  personnel  were  receptive  and  willing  to  participate  in  the 
study.  The  research  team  attempted  to  formulate  the  expert  panel  by  soliciting  personnel, 
possessing  the  required  qualifications,  from  various  major  commands  throughout  the  Air 
Force. 

Interviews 

A  short  electronic  interview  was  used  for  the  second  phase  of  research,  in  which  basic 
concepts  were  gathered  and  used  in  formulating  the  Delphi  questionnaire.  Personal  (face 
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to  face)  or  telephone  interviews  were  not  anticipated,  but  could  have  been  warranted  if 
time  permitted  and  the  electronic  interview  did  not  generate  satisfactory  results.  These 
secondary  interview  methods  have  the  same  advantages  in  high  participation  rates, 
interviewer  control,  and  flexibility;  however,  they  also  have  the  same  disadvantages  of 
interviewer  bias,  time  restrictions,  and  the  possibility  for  interpretation  error  (Parten, 
1966).  The  research  team  feels  that  electronic  interviews  provided  wide  geographical 
capability  at  a  low  cost  with  timely  feedback.  In  addition,  the  potential  for 
misinterpretation  was  significantly  reduced  with  the  written  document.  Furthermore, 
follow-up  questions  and  clarification  requests  were  also  faster  and  more  flexible  by  using 
electronic  media. 

The  interview  questions  covered  broad  categories  of  the  certification  model  and  were 
predominantly  very  general  in  nature.  The  interviews  were  comprised  of  eleven 
questions.  The  questions  were  worded  so  as  to  not  reference  or  lead  the  panel  members 
toward  thinking  about  the  existing  certification  models.  The  goal  here  was  to  draw  out 
new  and  innovative  ideas  that  can  be  applied  toward  contracting  certification.  Once  a 
framework  was  developed  from  these  initial  interviews,  more  specific  questions  were 
asked  in  the  form  of  a  Delphi  questionnaire. 

Delphi  Questionnaire 

The  members  of  the  expert  panel  were  each  asked  to  respond  to  a  questionnaire 
outlining  the  requirements  of  the  ideal  contracting  certification  model.  This  certification 
system  model  may  entail  two  or  three  levels  of  certification  (to  be  determined  in  the 
interviews).  Upon  receiving  the  initial  responses  from  the  interview,  the  research  team 
reviewed  and  consolidated  the  responses  and  included  the  group  modal  response  when 
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sending  out  the  second  round  of  questionnaire.  The  respondents  were  asked  to  reconsider 
each  answer  and  again  respond.  Additional  questions  were  added  to  the  second  round 
depending  upon  the  comments  received  on  the  first  round.  The  goal  of  the  research  team 
was  to  achieve  a  majority  consensus  (greater  than  50%)  among  experts  for  each  of  the 
questionnaire  items.  If  a  majority  consensus  was  not  achieved  after  two  rounds,  then  a 
third  and  possibly  fourth  round  of  questionnaire  was  conducted.  This  process  was 
repeated  until  the  panel  had  reached  a  majority  consensus.  Once  the  established  level  of 
agreement  was  attained,  the  research  team  presented  the  finished  product  -  the  ideal 
characterization  of  a  certification  system  for  the  contracting  profession. 

Model  Development  and  Stakeholder  Assessment 

The  research  team  compared  the  newly  constructed  model  with  the  two  existing 
contracting  certification  programs.  The  research  team  assessed  the  similarities  and 
differences  between  our  model  and  existing  certification  programs,  seeking  to  answer  the 
question  “How  do  they  measure  up  to  our  model?”  A  widespread  survey  was  developed 
and  implemented  to  assess  stakeholders’  perceptions  of  the  existing  certification 
programs  against  the  new  characterization.  A  combination  of  the  expert’s  input  and 
survey  results  assisted  in  identifying  possible  deficiencies  in  the  current  certification 
programs.  After  identifying  these  differences,  the  researchers  identified  possible 
corrective  actions  to  the  existing  programs  followed  by  recommendations  to  the 
contracting  profession  on  professional  certification. 
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IV.  Data  Description  and  Analysis 


Introduction 

This  chapter  describes  the  results  of  the  data  collection  phases  of  this  study.  The 
research  plan  described  in  Chapter  III  included  electronic  interviews,  a  Delphi  Survey, 
and  a  stakeholder  assessment.  The  purpose  of  the  electronic  interviews  was  to  use  open- 
ended  questions  to  elicit  freethinking  from  the  expert  panel  regarding  what  they 
considered  to  be  the  ideal  contracting  certification  model,  without  guiding  or  influencing 
them  towards  any  currently  existing  model.  The  objective  of  the  Delphi  Survey  was  to 
gain  a  consensus  from  the  expert  panel  to  formulate  the  ideal  contracting  certification 
model  based  upon  the  various  responses  to  the  electronic  interview.  The  final 
stakeholder  assessment  was  used  to  gain  insight  from  the  contracting  field  as  to  how  the 
ideal  model  is  perceived  when  compared  with  the  existing  certification  programs  offered 
byAPDPandNCMA. 

Each  of  the  thirteen  members  of  the  expert  panel  (listed  in  Appendix  A)  were  sent  the 
electronic  interview  and  the  Delphi  Questionnaire.  Of  the  thirteen  members  solicited,  10 
interviews  were  completed  and  returned,  and  1 1  members  responded  to  the  Delphi 
Questionnaire.  The  grades  of  the  civilian  panel  members  ranged  from  GS-13  to  members 
of  the  Selective  Executive  Service  (SES).  The  previously  stated  minimum  civilian  grade 
requirement  of  GS-14  was  relaxed  to  include  a  GS-13,  who  holds  multi-service 
contracting  experience.  This  is  a  highly  experienced  individual  in  both  the  Army  and  the 
Air  Force  and  is  considered  an  expert  by  the  research  team.  The  board  was  also 
comprised  of  four  Air  Force  Colonels,  and  a  civilian  from  both  the  Army  and  Navy.  The 
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research  team  feels  that  the  expert  panel  comprised  of  both  military  and  civilian 
contracting  officials,  from  three  branches  of  the  Department  of  Defense,  offered  a 
diversified  insight  to  the  formulation  of  the  ideal  contracting  certification  model. 

Initial  Interview  in  the  Formulation  of  the  Ideal  Contracting  Certification  Model 
Interview  Questions.  The  interview  questions  are  listed  in  Appendix  B.  The 
interviews  consisted  of  six  questions  designed  to  get  the  expert  panel  to  express  their 
thoughts  and  rationale  towards  an  ideal  contracting  certification  program.  Two  questions 
asked  the  panel  members  to  make  a  choice,  either  yes  or  no,  but  to  also  indicate  their 
rationale  behind  their  decision.  The  other  four  questions  were  open-ended  and  asked  the 
panel  members  to  describe  what  they  would  view  as  the  ideal  certification  program. 

General  Results.  The  results  of  the  interview  process  provided  the  framework  for  the 
Delphi  questionnaire,  and  were  considered  by  the  research  team  adequate  to  serve  as  the 
first  round  of  the  Delphi  process.  Each  interview  question,  as  anticipated,  generated 
various  responses  from  the  expert  panel.  Although  some  responses  were  more  prevalent 
than  others,  the  research  team  determined  no  single  response  represented  a  strong 
consensus  among  panel  members. 

Results  and  Expert  Comments  bv  Question.  The  overall  response  for  each  of  the  six 
questions,  as  well  as  specific  comments  provided  by  the  panel  members,  are  identified 
below.  The  specific  comments  listed  throughout  this  chapter  are  actual  quotes  from 
members  of  the  expert  panel.  However,  due  to  the  anonymity  of  the  Delphi  process,  as 
discussed  in  Chapter  III,  the  quotes  are  not  referenced. 
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Question  1.  Should  the  ideal  contracting  certification  program  have  multiple 
levels  of  certification?  Why  or  why  not?  If  yes,  how  many  levels  do  you  believe  would 
be  appropriate  and  why? 

Overall  Response.  While  80%  of  the  panel  members  were  in  favor  of 
multiple  certification  levels,  they  did  not  reach  consensus  concerning  the  ideal  number  of 
certification  levels.  40%  of  the  panel  agreed  that  three  levels  would  be  ideal,  30% 
believed  that  two  levels  would  be  ideal,  10%  felt  that  four  levels  would  be  ideal,  and  20% 
indicated  that  multiple  levels  are  certification  were  not  necessary.  Comments  supporting 
each  of  these  positions  follow. 

Specific  Comments.  In  not  favoring  multiple  levels  of  certification,  one 
panel  member  responded  “multiple  levels  cheapen/degrade  the  prestige  of 
certification... CPA’s  don’t  have  an  ‘almost  good  enough  to  be  a  CPA’  certificate.” 

While  this  statement  is  respected  by  the  research  team,  it  is  considered  to  be  a  minority 
opinion  with  respect  to  the  research  panel,  and  was  not  included  in  the  follow-on  Delphi 
survey.  In  supporting  two  levels  of  certification,  a  panel  member  stated  “A  certification 
program  with  multiple  levels  that  is  properly  implemented  can  provide  a  mechanism  for 
both  recognition  and  motivation.  The  number  of  levels  should  not  exceed  two.”  In 
contrast  to  the  two  level  approach,  many  panel  members  stated  that  three  levels  would  be 
more  ideal.  In  supporting  three  levels  of  certification,  one  panel  member  stated  “the  3- 
level  structure  tracks  to  the  logical  categorization  based  upon  individual  capability  and 
experience.  Two  levels  would  be  too  little  and  four  levels  would  be  greater  than  the 
degree  of  distinction  recognizable  in  the  workforce.”  This  was  the  most  common 
response  for  this  question  gaining  a  40%  response  rate  from  the  expert  panel.  Finally, 
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one  panel  member  supported  a  fourth  certification  level  by  stating  “Presently,  a  Level  III 
(APDP)  can  be  obtained  very  early.  This  is  the  last  certification  an  individual  needs  to  be 
concerned  about.  A  fourth  certification  should  be  obtained  with  about  8  years  in  the  field 
to  allow  the  individual  to  continue  to  stress  their  own  development  in  the  field.”  This 
was  considered  by  the  research  team  to  be  a  minority  opinion  among  the  expert  panel, 
and  was  not  included  in  the  follow-on  Delphi  questionnaire. 

Conclusion.  Based  upon  the  responses  provided,  the  research  team 
concluded  that  the  expert  panel  reached  a  majority  consensus  in  regards  to  favoring 
multiple  levels  of  certification.  However,  among  the  panel  members  that  opted  for 
multiple  levels,  a  consensus  was  not  reached  with  regard  to  the  exact  number  of  levels. 
Thus,  the  research  team  will  include  this  question  again  in  a  Delphi  questionnaire  that 
will  limit  the  choices  of  the  expert  panel.  The  expert  panel  will  be  asked  to  choose 
between  two  or  three  certification  levels  for  the  ideal  model.  While  the  research  team 
respects  the  arguments  for  both  a  single  level  as  well  as  a  four  level  system,  it  was 
determined  that  these  responses  were  unlikely  to  gain  a  timely  consensus  from  the  expert 
panel,  and  were  thus  not  included  in  the  formulation  of  the  follow-on  questionnaire. 

Question  2.  How  many  years  of  contracting  experience  should  an  individual  have 
to  qualify  for  certification?  Please  identify  the  years  of  experience  required  with  each 
level  of  certification  identified  in  Question  1 . 

Overall  Response.  The  responses  to  this  question  varied  tremendously. 
Under  a  two-level  system,  the  panel  members  were  evenly  split  between  three,  four,  and 
five  years  of  experience  for  the  first  level.  For  the  second  level  of  a  two-level  system, 

33%  of  the  panel  recommended  seven  years  of  experience,  while  67%  of  the  panel 
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preferred  ten  years.  Under  a  three-level  system,  50%  of  the  panel  agreed  that  three  years 
should  be  the  minimum  experience  requirement  for  the  first  level  of  certification.  25%  of 
the  panel  stated  two  years  would  be  sufficient,  and  25%  stated  that  no  experience  (entry 
level)  should  be  required  for  the  first  level.  For  the  second  level  of  a  three-level  system, 
the  panel  was  evenly  split  among  three,  four,  five,  and  six  years  of  contracting 
experience.  Finally,  for  the  third  level  of  a  three-level  system,  50%  of  the  panel  agreed 
that  ten  years  should  be  the  minimum  experience  requirement  to  reach  the  highest  level 
of  certification.  25%  of  the  panel  stated  eight  years  would  be  sufficient,  and  25%  stated 
that  a  minimum  of  five  years  would  be  sufficient  to  reach  this  level. 

Specific  Comments.  Very  few  comments  were  received  in  response  to 
this  question.  The  majority  of  the  expert  panel  simply  responded  with  the  years  of 
contracting  experience  they  felt  would  be  commensurate  with  the  certification  levels  they 
indicated  in  Question  1 .  However,  in  support  of  using  experience  as  a  certification 
credential,  one  panel  member  stated  “Experience  and  education  are  critical  components 
of  any  certification  program.”  This  statement  appears  to  be  supported  by  the  majority  of 
the  expert  panel,  based  upon  the  experience  requirements  in  the  above  responses. 
However,  one  panel  member  warned  against  simply  using  time  as  a  measure  of 
experience  stating  that  it  is  hard  to  determine  if  a  person  has  “20  years  of  experience  or 
one  year  of  experience  20  times  over.”  Another  panel  member  supported  this  thought 
stating  “Do  not  tie  certification  to  experience  unless  you  can  also  measure  the  breadth 
and  depth  of  experience.”  The  research  team  acknowledges  that  mere  time  on  the  job  is 
not  necessarily  meaningful  experience,  but  also  feels  that  a  certain  amoimt  of  experience 
should  be  required  before  an  individual  can  be  certified  in  any  profession.  The 
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importance  of  the  experience  factor  is  echoed  by  90%  of  the  expert  panel  with  regard  to 
their  responses,  and  is  considered  to  be  a  critical  component  of  the  certification  process. 

Conclusion.  Based  upon  the  responses  provided,  the  research  team 
concluded  that  the  expert  panel  reached  a  majority  consensus  in  regards  to  including 
contracting  experience  in  the  ideal  certification  model.  However,  a  consensus  was  not 
reached  on  the  exact  amount  of  experience  necessary  for  each  level  of  certification. 

Thus,  the  research  team  will  include  this  question  again  in  a  Delphi  questionnaire  that 
will  limit  the  choices  of  the  expert  panel.  The  expert  panel  will  be  asked  to  select  the 
appropriate  experience  criteria  for  each  level  in  both  a  two-level  and  a  three-level 
program.  For  the  first  level  of  a  two-level  program,  the  options  will  be  three,  four  or  five 
years.  For  the  second  level  of  a  two-level  program,  the  options  will  be  either  seven  or  ten 
years.  For  the  first  level  of  a  three-level  program,  the  options  will  be  zero,  two  or  three 
years.  For  the  second  level  of  a  three-level  program,  the  options  will  be  three,  four,  five 
or  six  years.  Finally,  for  the  third  level  of  a  three-level  program,  the  options  will  be  five, 
eight  or  ten  years.  Only  the  responses  given  in  the  interviews  will  be  provided  as  options 
for  the  Delphi  questionnaire.  This  reduces  the  influence  that  the  research  team  has  in  the 
formulation  of  the  ideal  certification  model. 

Question  3.  What  levels  of  education  should  an  individual  have  to  qualify  for 
certification?  Please  identify  the  education  level  required  with  each  level  of  certification 
identified  in  Question  1 . 

Overall  Response.  For  those  panel  members  favoring  a  two-level  system, 
100%  agreed  that  a  Bachelor’s  degree  should  be  required  for  the  first  level  of 
certification.  67%  of  the  panel  who  favored  a  two-level  system,  agreed  that  a  Master’s 
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degree  should  be  required  to  achieve  the  second  level,  and  33%  stated  that  a  professional 
certification  such  as  NCMA  should  be  required  to  achieve  the  second  level.  Under  a 
three-level  system,  50%  of  the  panel  agreed  that  a  Bachelor’s  degree  should  be  required 
for  all  three  levels,  while  the  other  50%  stated  that  education  levels  were  not  relevant  to 
the  certification  process.  30%  of  the  panel  members  stated  that  a  Master’s  degree  should 
be  desired  for  the  third  level  of  a  three-level  system,  but  should  not  be  required. 

Specific  Comments.  Very  few  comments  were  received  in  response  to 
this  question.  The  majority  of  the  expert  panel  simply  responded  with  the  level  of 
education  they  felt  would  be  commensurate  with  the  certification  levels  identified  in 
Question  1 .  However,  in  regard  to  education  levels,  one  panel  member  stated  “(Do)  not 
tie  the  degree  to  a  specific  field  such  as  Business,  but  use  the  degree  as  an  indicator  of 
potential.”  Other  panel  members  stated  that  the  current  DAWIA  requirements  were 
satisfactory  calling  for  a  Bachelor’s  degree  or  24  hours  of  business  classes.  These 
comments  appear  consistent  in  that  a  Bachelor’s  degree  in  any  field  should  deem  a  person 
eligible  for  certification.  However,  several  panel  members  felt  that  education  is  not  a 
relevant  part  of  the  certification  process.  One  panel  member  stated  “I  have  not  seen  a 
direct  relationship  between  education  level  and  an  individual’s  ability  to  do  the  job”, 
another  panel  member  echoed  this  statement  in  that  he  felt  that  longer  experience  time  is 
a  suitable  substitute  for  business  education.  Thus,  a  majority  consensus  was  not  reached 
as  to  the  education  levels  required  for  contracting  certification  under  the  ideal  model. 

Conclusion.  Based  upon  the  responses  provided,  the  research  team 
concluded  that  the  expert  panel  did  not  reach  a  majority  consensus  in  regards  to  including 
education  levels  in  the  ideal  certification  model.  As  a  result,  the  research  team  will 
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include  this  question  again  in  a  Delphi  questionnaire  that  will  limit  the  choices  of  the 
expert  panel.  The  expert  panel  will  be  asked  to  select  the  appropriate  education  level  for 
eaeh  certification  level  in  both  a  two-level  and  a  three-level  program.  For  the  first  level 
of  a  two-level  program,  the  options  will  be  either  a  Bachelor’s  degree  or  “other”  in  which 
the  panel  members  can  elaborate  or  specify  an  alternate  education  preference.  For  the 
second  level  of  a  two-level  program,  the  options  will  be  Master’s  degree.  Additional 
Professional  Certification,  or  “other”.  For  each  level  in  a  three-level  program,  the  options 
will  be  either  Bachelor’s  degree  or  “Education  levels  are  not  relevant”.  These  were  the 
only  two  responses  identified  in  the  interviews  for  the  three-level  program.  Again,  only 
the  responses  given  in  the  interviews  will  be  provided  as  options  for  the  Delphi 
questionnaire. 

Question  4.  What  kinds  of  contracting  related  courses  (fundamentals,  law, 
negotiation,  etc.)  should  an  individual  complete  prior  to  being  certified?  Should  different 
types  of  courses  correspond  to  different  levels  of  certification? 

Overall  Response.  WTiile  the  intent  of  this  question  was  for  the  panel 
members  to  identify  specific  courses  and  relate  these  courses  to  the  various  levels  of 
certification,  very  few  panel  members  responded  in  this  manner.  The  majority  of 
responses  were  not  specific  enough  for  the  research  team  to  identify  the  contracting 
course  work  that  should  be  required  for  certification  under  an  ideal  model. 

Specific  Comments.  30%  of  responses  included  an  opinion  that  the 
existing  APDP  course  requirements  were  sufficient  for  a  contracting  certification 
program.  WTiile  the  existing  APDP  coursework  may  be  sufficient,  other  panel  members 
indicated  that  the  existing  requirements  could  be  improved  by  requiring  at  least  one  more 
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course  every  two  years,  or  by  incorporating  specialty  courses  based  upon  an  individual’s 
specific  assignment.  20%  of  the  panel  actually  identified  which  courses  should  be 
required  to  reach  each  level  of  certification.  While  these  panel  members  predominantly 
listed  the  existing  APDP  curriculum,  they  also  indicated  that  some  courses  should  be 
required  more  early  on  (i.e.,  level  one)  than  are  currently  required. 

Conclusion.  As  a  result  of  the  generic  responses  received,  a  majority 
consensus  was  not  reached  as  to  which  courses  should  be  required  for  the  specific  levels 
of  certification.  Thus,  the  research  team  will  attempt  to  gain  more  specific  responses  by 
including  this  question  again  in  the  Delphi  questionnaire.  The  questionnaire  will  list  all 
the  courses  identified  in  the  interview  process  and  ask  the  panel  members  to  indicate 
which  level  of  certification  they  should  be  required  for  under  both  a  two-level  and  a 
three-level  system.  The  panel  members  will  have  the  option  of  choosing  a  certification 
level  (one  through  three)  or  indicating  that  a  specific  course  is  not  relevant.  Using  this 
approach,  the  research  team  expects  to  obtain  a  clear  consensus  as  to  which  courses 
should  be  required  for  the  various  levels  of  certification.  Only  the  courses  identified  by 
panel  members  in  the  interview  process  will  appear  on  the  questionnaire.  Again,  this  is 
designed  to  reduce  the  influence  that  the  research  team  has  in  the  formulation  of  the  ideal 
certification  model. 

Question  5.  Should  certification  imder  the  ideal  contracting  certification  program 
require  an  individual  to  pass  a  comprehensive  examination?  Why  or  why  not? 

Overall  Response.  70%  of  the  panel  responded  favorably  to  the 
incorporation  of  a  comprehensive  examination  into  the  ideal  contracting  certification 
program.  Of  this  70%,  60%  indicated  that  a  comprehensive  examination  should  be 
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administered  for  all  levels  of  eertification,  while  10%  felt  that  a  comprehensive 
examination  should  only  be  required  for  the  highest  level  of  certification.  30%  of  the 
panel  felt  that  a  comprehensive  examination  would  not  be  beneficial  to  the  ideal  model. 

Specific  Comments.  Advocates  of  a  comprehensive  examination  stated 
such  reasons  as  keeping  standards  high,  keeping  abreast  of  changes  and  ensuring  a 
demonstrated  knowledge  as  reasons  why  a  comprehensive  examination  would  be 
beneficial.  One  panel  member  captured  this  view  in  stating  “. .  .this  keeps  the  individual 
thinking  and  staying  abreast  of  the  changes  in  the  field  so  they  can  respond  to  questions 
pertinent  to  the  field.”  Even  though  advocating  a  comprehensive  examination,  certain 
panel  members  warned  of  the  inherent  challenges.  One  panel  member  cautioned  the 
implementation  of  such  an  examination  by  stating  “There  are,  of  course,  problems  with 
tests;  who  prepares,  administers  and  scores  the  test  is  the  biggest  challenge.”  30%  of  the 
panel  members  were  opposed  to  such  an  examination  stating  the  administrative  burden 
and  the  danger  of  people  memorizing  things  solely  for  the  purpose  of  test  taking  as  the 
driving  reasons.  One  panel  member  in  opposition  to  a  comprehensive  examination  stated 
“I  do  not  want  contracting  people  memorizing  things  for  tests.  I  want  them  in  the 
regulations  to  insure  they  know  what  they  are  doing.”  Another  panel  member  also  had  a 
valid  point  in  identifying  reasons  against  implementing  a  comprehensive  examination 
stating  "I  believe  the  only  valid  examination  would  have  to  be  scenario  type  questions. 
The  administrative  burden  of  developing  a  pool  of  questions  to  use  (of  equal  difficulty) 
and  of  scoring  such  a  test  is  prohibitive.” 

Conclusion.  Based  upon  the  responses  provided,  the  research  team 
concluded  that  the  expert  panel  reached  a  majority  consensus  in  regards  to  including  a 
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comprehensive  examination  in  the  ideal  certification  model.  However,  the  desired 
consensus  level  was  not  reached  to  identify  for  which  certification  levels  a  test  should  be 
required.  Thus,  the  research  team  will  include  this  question  again  in  a  Delphi 
questionnaire  that  will  limit  the  choices  of  the  expert  panel.  The  expert  panel  will  again 
be  asked  if  a  comprehensive  examination  should  be  required  for  certification.  Three 
options  will  be  provided  for  the  panel  to  choose  from:  1)  Yes,  for  all  levels,  2)  Yes,  for 
only  the  most  advanced  level,  and  3)  No,  an  examination  would  not  be  beneficial.  Using 
this  form  of  question,  the  research  team  seeks  to  gain  a  clear  consensus  on  the  use  of  a 
comprehensive  examination  in  the  ideal  contracting  certification  model.  Only  the 
responses  given  in  the  interviews  will  be  provided  as  options  for  the  Delphi 
questionnaire.  This  reduces  the  influence  that  the  research  team  has  in  the  formulation  of 
the  ideal  certification  model. 

Question  6.  Under  an  ideal  contracting  certification  program,  should  periodic  re¬ 
certification  be  required  for  an  individual  to  retain  his/her  certification?  Why  or  why 
not?  If  you  believe  re-certification  should  be  required,  how  often  should  this  be 
accomplished  and  what  should  it  entail? 

Overall  Response.  A  strong  majority  (70%)  of  the  panel  agreed  that  re¬ 
certification  should  be  incorporated  into  the  ideal  contracting  certification  program.  Of 
this  70%,  50%  indicated  that  re-certification  should  be  based  upon  experience  and 
continuing  education,  while  10%  felt  that  a  periodic  examination  should  be  required  for 
re-certification.  The  remaining  10%  felt  that  re-certification  should  be  based  upon 
additional  experience,  continuing  education,  and  a  periodic  examination.  In  addition, 
30%  of  the  panel  felt  that  re-certification  should  not  be  included  in  the  ideal  model. 
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Specific  Comments.  In  support  of  a  re-certification  requirement,  one 
panel  member  stated  “Federal  acquisition  has  become  so  complex  that  it  is  necessary  to 
make  sure  we  are  keeping  the  right  people  with  the  right  level  of  imderstanding  in  the 
right  positions. . .  A  recertification  test  should  cover  current  issues  and  pose  practical 
problems.”  While  a  re-certification  test  was  only  favored  by  10%  of  the  panel,  other 
panel  members  favored  re-certification  based  upon  experience  and  continuing  education. 
As  one  panel  stated  “Re-certification  should  be  required  every  5  years  to  reinforce  the 
meaning  and  significance  of  certification.  ..An  individual  should  complete  an  average  of 
40  hours  of  specialized  training  per  year  over  the  past  5-year  period.”  Other  panel 
members  recommended  a  continuing  education  requirement  of  three  courses  every  four 
years  to  be  eligible  for  re-certification.  However,  30%  of  the  panel  was  opposed  to  a  re¬ 
certification  requirement.  A  recurring  theme  among  these  panel  members  was  that  once 
an  individual  obtains  a  certain  level  of  certification  they  should  be  able  to  maintain  that 
level  similar  to  being  granted  a  contracting  warrant.  One  panel  member  further  cautioned 
against  a  re-certification  program  by  stating  that  the  Government’s  grievance  process 
would  significantly  reduce  a  supervisor’s  ability  to  deny  re-certification  to  a  satisfactory 
performer. 

Conclusion.  Based  upon  the  responses  provided,  the  research  team 
concluded  that  the  expert  panel  reached  a  majority  consensus  in  regards  to  including  a  re¬ 
certification  requirement  in  the  ideal  certification  model.  However,  the  desired 
consensus  level  was  not  reached  to  identify  which  criteria  should  be  met  in  order  for  an 
individual  to  qualify  for  re-certification.  Thus,  the  research  team  developed  three  follow- 
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on  questions  to  be  used  in  a  Delphi  questionnaire  which  will  limit  the  choices  of  the 
expert  panel. 

First,  the  expert  panel  will  be  asked  if  periodic  re-certification  should  be  required 
under  the  ideal  program.  Four  options  will  be  provided  for  the  panel  to  choose  from:  1) 
Yes,  via  an  examination,  2)  Yes,  via  experience  and  continuing  education,  3)  Yes,  via  an 
examination,  experience,  and  continuing  education,  and  4)  No,  re-certification  should  not 
be  required.  Using  this  form  of  question,  the  research  team  seeks  to  gain  a  clear 
consensus  on  the  criteria  with  which  to  incorporate  or  not  incorporate  a  re-certification 
requirement. 

Secondly,  with  regard  to  re-certification,  the  expert  panel  will  be  asked  to  select  an 
appropriate  time  frame  for  which  a  certification  remains  valid.  The  expert  panel  will  be 
able  to  choose  in  one-year  increments  from  one  year  up  to  five  years.  One  year  and  five 
years  were  the  two  extreme  points  identified  in  the  interview  process  with  regard  to  the 
re-certification  timeframe  and  as  a  result  are  used  as  a  range  for  the  questioimaire 
responses.  Using  this  form  of  question,  the  research  team  seeks  to  gain  a  clear  consensus 
as  to  the  validity  period  granted  each  certification  under  the  ideal  model. 

Finally,  with  regard  to  the  continuing  education  requirement,  the  expert  panel  will  be 
asked  to  identify  the  proper  amoimt  of  annual  education  (on  average)  that  should  be 
required  for  re-certification.  The  following  options  will  be  available  for  the  expert  panel 
to  select  from:  1)  30  hours  per  year,  2)  40  hours  per  year,  and  3)  “Other”  in  which  the 
panel  member  will  be  asked  to  enter  an  exact  number  of  hours  per  year.  The  first  two 
options  were  taken  directly  from  the  interview  responses.  However,  as  only  30%  of  the 
expert  panel  responded  to  the  interview  with  an  exact  continuing  education  requirement, 
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the  research  team  included  a  third  option  for  this  question  in  which  the  panel  members 
will  be  permitted  to  enter  their  own  criteria.  Using  this  form  of  question,  the  research 
team  seeks  to  gain  a  clear  consensus  on  the  continuing  education  criteria  regarding  re¬ 
certification  under  the  ideal  contracting  certification  program. 

Delphi  Questionnaire  and  Model  Formulation 

Delphi  Questions.  The  Delphi  questionnaire  used  in  this  phase  of  data  collection  is 
provided  in  Appendix  C.  The  questions  used  for  this  instrument  were  derived  from  the 
electronic  interviews  discussed  in  the  previous  section.  The  rationale  for  the  inclusion  of 
each  question,  the  question  format,  and  the  provided  choices  was  provided  in  the  analysis 
of  the  interview  responses  in  the  previous  section.  The  questionnaire  consisted  of  eleven 
multiple-choice  questions  designed  to  obtain  a  consensus  among  panel  members  as  to  the 
precise  characteristics  and  requirements  for  the  ideal  contracting  certification  model. 

General  Results.  The  results  of  the  Delphi  questionnaire  provided  for  the  formulation 
of  the  ideal  contracting  certification  model.  Of  the  13  expert  panel  members,  1 1 
members  responded  to  this  questionnaire.  While  the  research  team  originally  sought  to 
reach  a  70%  consensus  on  each  aspect  of  the  certification  model,  the  research  team 
relaxed  the  consensus  goal  to  a  simple  majority  decision  on  each  element  of  the  model  to 
facilitate  the  timeline  of  this  research  effort.  By  relaxing  the  70%  consensus  goal  to  a 
simple  majority  decision,  the  research  team  was  able  to  complete  the  Delphi  portion  of 
this  research  with  only  two  rounds  of  data  collection,  and  was  then  able  to  conduct  a 
thorough  stakeholder  assessment  of  the  formulated  model. 

Results  by  Question  and  Model  Formulation.  The  overall  responses  for  each  of  the 
eleven  questions  are  identified  below.  For  each  question,  the  response  receiving  the 
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majority  consensus  was  incorporated  into  the  ideal  contracting  certification  model.  The 
majority  response  to  some  questions  made  other  questions  irrelevant  in  the  formulation  of 
the  ideal  model.  For  example,  Question  1  asked  the  panel  to  select  from  either  a  two- 
level  program  or  a  three-level  program.  Since  the  three-level  program  received  the 
majority  response  rate,  the  questions  pertaining  exclusively  to  a  two-level  system 
(Questions  2, 4,  and  6)  were  no  longer  pertinent  in  the  formulation  of  the  ideal  model. 
The  results  to  these  questions  will  still  be  provided  below  for  informational  purposes,  but 
are  not  included  in  the  model  formulation.  This  method  was  designed  by  the  research 
team  to  save  time  in  the  data  collection  phase  of  this  research.  By  asking  questions 
tailored  to  both  a  two-level  and  a  three-level  system  simultaneously,  the  research  team 
had  the  data  available  to  support  either  system.  This  was  designed  to  prevent  the  need  for 
future  rounds  of  the  Delphi  process.  The  results  from  the  Delphi  questionnaire  are 
provided  below. 

Question  1  -  Levels  of  Certification.  How  many  levels  of  certification  do  you 
believe  would  be  appropriate  for  the  ideal  contracting  certification  program? 

Response.  45%  Two  Levels 

55%  ThreeLevels 

Analysis.  The  expert  panel  preferred  a  three-level  certification 
program  as  opposed  to  a  two-level  certification  program  by  a  very  small  margin.  Despite 
this  small  margin,  the  research  team  was  convinced  that  future  rounds  of  Delphi  surveys 
would  not  neeessarily  change  the  majority  opinion  in  regards  to  this  question.  As  a 
result,  and  adhering  to  the  majority  opinion  of  the  expert  panel,  the  research  team 
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concluded  that  the  ideal  contracting  certification  program  should  be  comprised  of  three 
distinct  levels  of  certification. 

Question  2  -  Experience  (two-level).  How  many  years  of  contracting  experience 
should  an  individual  have  to  possess  in  order  to  qualify  for  certification  under  a  TWO- 
LEVEL  program?  Note;  based  upon  the  majority  response  to  question  one,  a  two-level 
program  was  not  adopted  for  the  ideal  model.  Nonetheless,  the  results  to  this  question 
are  provided  for  information  purposes  only. 

Response.  LEVEL  ONE: 

91%  Three  Years 


9% 

Four  Years 

0% 

Five  Years 

LEVEL  TWO; 

73% 

Seven  Years 

27% 

Ten  Years 

Analysis.  As  stated  above,  this  question  was  determined  to  be  irrelevant 
for  the  ideal  model  based  upon  the  panels  selection  of  a  three-level  system  in  Question  1 
Nonetheless,  it  is  interesting  to  note  that  if  a  two-level  system  had  been  adopted,  a  clear 
consensus  existed  that  individuals  should  possess  a  minimum  of  three  years  contracting 
experience  to  reach  the  first  level,  and  a  total  of  seven  years  contracting  experience  to 
reach  the  second  level.  The  experience  requirements  for  the  selected  three-level  system 
will  be  identified  in  the  following  question. 
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Question  3  -  Experience  (three-level  V  How  many  years  of  contracting  experience 
should  an  individual  have  to  possess  in  order  to  qualify  for  certification  under  a  THREE- 
LEVEL  program? 

Response.  LEVEL  ONE: 

9%  No  experience 

55%  Two  Years 

36%  Three  Years 

LEVEL  TWO: 

9%  Three  Years 

27.5%  Four  Years 

36%  Five  Years 

27.5%  Six  Years 

LEVEL  THREE: 

0%  Five  Years 

27%  Eight  Years 

73%  Ten  Years 

Analysis.  This  question  constructs  a  critical  component  of  the  ideal 
certification  model,  in  that  specific  experience  requirements  are  identified  for  each  of  the 
three  levels  of  certification.  The  majority  of  panel  members  (55%)  concurred  that  two 
years  of  contracting  experience  should  be  required  before  an  individual  can  be  certified  at 
the  first  level  (entry-level)  of  certification.  The  experience  level  required  to  reach  the 
second  level  of  certification  was  determined  by  the  expert  panel  to  be  five  years.  While 
the  panel  was  nearly  split  between  four,  five,  and  six  years  of  experience  for  this  second 
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level  of  certification,  five  years  of  experience  received  slightly  more  support  (36%)  from 
the  expert  panel,  and  was  thus  adopted  for  the  ideal  model.  The  panel  reached  a  decisive 
consensus  with  regard  to  the  third  level  of  certification,  with  73%  of  the  panel  agreeing 
that  1 0  years  of  contracting  experience  should  be  required  before  an  individual  can 
possess  the  highest  contracting  certification  level.  These  experience  requirements  will  be 
compared  with  the  existing  APDP  and  NCMA  requirements  during  the  stakeholder 
assessment  portion  of  this  chapter. 

Question  4  -  Education  (two-level).  What  level  of  education  should  an  individual 
have  to  possess  in  order  to  qualify  for  certification  under  a  TWO-LEVEL  program? 

Response.  LEVEL  ONE: 

91%  B.S./B.A. 

9%  Education  level  is  not  relevant 

LEVEL  TWO: 

45.5%  Masters  Degree 
36.5%  Additional  Professional  Certification 

9%  A  broad  based  specialty  training  program 

9%  No  experience 

Analysis.  As  stated  above,  this  question  was  determined  to  be  irrelevant 
for  the  ideal  model  based  upon  the  panels  selection  of  a  three-level  system  in  Question  1 . 
Nonetheless,  it  is  interesting  to  note  that  if  a  two-level  system  had  been  adopted,  a  clear 
consensus  (91%)  existed  that  individuals  should  possess  a  Bachelors  degree  in  order  to 
qualify  for  the  first  level  of  certification.  In  addition,  the  expert  panel  gave  a  slight 
preference  (45.5%)  towards  requiring  a  Masters  degree  in  order  to  reach  the  second  level. 
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The  educational  requirements  for  the  selected  three-level  system  will  be  identified  in  the 
following  question. 

Question  5  -  Education  ('three-levelT  What  level  of  education  should  an  individual 
have  to  possess  in  order  to  qualify  for  certification  under  a  THREE-LEVEL  program? 
Response.  LEVEL  ONE: 

73%  B.S./B.A. 

27%  Education  level  is  not  relevant 
LEVEL  TWO: 

91%  B.S./B.A. 

9%  Education  level  is  not  relevant 

LEVEL  THREE: 

73%  B.S./B.A. 

9%  Education  level  is  not  relevant 

18%  Masters  Degree  (write-in  response) 

Analysis.  This  question  constructs  a  critical  component  of  the  ideal 
certification  model,  in  that  specific  educational  requirements  are  identified  for  each  of  the 
three  levels  of  certification.  A  strong  consensus  existed  between  panel  members  (>  73%) 
that  a  Bachelors  degree  should  be  required  for  each  of  the  three  certification  levels.  It  is 
interesting  that  although  Masters  degree  was  not  one  of  the  options  provided,  1 8%  of  the 
panel  felt  compelled  to  write-in  this  response.  As  identified  above,  the  options  provided 
in  response  to  this  question  were  taken  directly  from  the  interviews  conducted  in  phase  2 
of  this  research.  No  panel  member  advocating  a  three-level  system  indicated  a  desire  for 
a  Masters  degree  requirement,  and  thus  this  option  was  not  provided  on  the  questionnaire. 
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Since  18%  of  the  panel  felt  compelled  to  write-in  this  response,  perhaps  if  a  Masters 
degree  option  was  provided  on  the  questionnaire,  more  panel  members  would  have  voted 
in  favor  of  the  higher  educational  requirement.  Nonetheless,  the  research  team  accepted 
the  strong  consensus  of  the  panel  and  implemented  a  Bachelors  degree  requirement  for 
each  of  level  of  certification  in  the  ideal  model.  This  educational  requirement  will  be 
compared  with  the  existing  APDP  and  NCMA  requirements  during  the  stakeholder 
assessment  portion  of  this  chapter. 

Question  6  -  Formal  Training  (two-leveP.  Please  indicate  to  which  level  of 
certification  the  following  contracting  courses  should  be  required  under  a  TWO-LEVFJ. 
program. 

Response. 

Contracting  Fundamentals: 

100%  Level  One  0%  Level  Two  0%  Not  Relevant 

Contract  Pricing: 

100%  Level  One  0%  Level  Two  0%  Not  Relevant 

Specialty  Course  (Depending  on  Assignment): 

36%  Level  One  64%  Level  Two  0%  Not  Relevant 

Intermediate  Contracting: 

_ ^7%  Level  One  73%  Level  Two  0%  Not  Relevant 

Intermediate  Pricing: 

9%  Level  One  91%  Level  Two  0%  Not  Relevant 

Contract  Law: 

82%  Level  One  18%  Level  Two  0%  Not  Relevant 
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Contract  Negotiation: 

100%  Level  One  0%  Level  Two  0%  Not  Relevant 
Executive  Contracting: 

0%  Level  One  82%  Level  Two  18%  Not  Relevant 

Advanced  Topics  in  Acquisition: 

0%  Level  One  82%  Level  Two  18%  Not  Relevant 

Management  of  Contracting  Activities: 

0%  Level  One  82%  Level  Two  18%  Not  Relevant 

Contract  Process  Improvements: 

9%  Level  One  73%  Level  Two  18%  Not  Relevant 

Analysis.  As  stated  above,  this  question  was  determined  to  be  irrelevant 
for  the  ideal  model  based  upon  the  panel’s  selection  of  a  three-level  system  in  Question  1. 
Nonetheless,  it  is  interesting  to  note  that  if  a  two-level  system  had  been  adopted,  a 
majority  consensus  (>55%)  existed  as  far  as  which  courses  should  be  required  for  each 
level  of  certification.  It  is  also  interesting  to  note  that  only  a  minority  of  panel  members 
(<18%)  stated  that  some  courses  were  not  relevant  to  the  certification  process.  This  gives 
strong  support  for  the  importance  of  continuing  education  in  the  contracting  field.  The 
desired  coursework  for  the  selected  three-level  system  will  be  identified  in  the  following 
question. 

Question  7  -  Formal  Training  fthree-leveli.  Please  indicate  to  which  level  of 
certification  the  following  contracting  courses  should  be  required  under  a  THREE- 
LEVEL  program. 
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Contracting  Fundamentals: 

100%  Level  1  0%  Level  2  0%  Level  3  0%  N/R 

Contract  Pricing: 

100%  Level  1  0%  Level  2  0%  Level  3  0%  N/R 

Specialty  Course  (Depending  on  Assignment): 

27%  Level  1  64%  Level!  9%  Level 3  0%  N/R 

Intermediate  Contracting: 

9%  Level  1  91%  Level  2  0%  Level  3  0%  N/R 

Intermediate  Pricing: 

0%  Level  1  91%  Level  2  9%  Level  3  0%  N/R 

Contract  Law: 

55%  Level  1  45%  Level  2  0%  Level  3  0%  N/R 

Contract  Negotiation: 

73%  Level  1  27%  Level  2  0%  Level  3  0%  N/R 

Executive  Contracting: 

_ 0%  Level  1  0%  Level 2  _91%_  Levels  9%  N/R 

Advanced  Topics  in  Acquisition: 

0%  Level  1  18%  Level  2  73%  Level  3  9%  N/R 

Management  of  Contracting  Activities: 

0%  Level  1  18%  Level  2  73%  Level  3  9%  N/R 

Contract  Process  Improvements: 

9%  Level  1  27%  Level  2  45%  Level  3  18%  N/R 
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Analysis.  This  question  constructs  a  critical  component  of  the  ideal 
certification  model,  in  that  specific  contracting  course  requirements  are  identified  for 
each  of  the  three  levels  of  certification.  The  above  results  are  summarized  in  Table  6. 


Table  6.  Procurement  Related  Courses  Required  for  the  Ideal  Certification  Model 


Certification  Level 

Required  Courses 

Level  1 

Contracting  Fundamentals,  Contract  Pricing,  Contract  Law, 
Contract  Negotiation 

Level  2 

Intermediate  Contracting,  Intermediate  Pricing,  Specialty 

Course 

Level  3 

Executive  Contracting,  Advanced  Topics  in  Acquisition, 
Management  of  Contracting  Activities,  Contract  Process 
Improvements 

The  panel  reached  a  strong  consensus  (>73%)  for  all  the  listed  courses  with  the 
exception  of  three.  First,  the  panel  reached  a  narrow  majority  decision  with  regards  to 
the  specialty  course  identified  for  Level  2.  This  is  possibly  due  to  the  fact  that  more 
specific  information  was  not  provided  to  the  expert  panel  as  to  what  exactly  the  specialty 
course  contained.  The  specialty  course  was  identified  by  a  panel  member  during  the 
interview  process,  and  was  thus  provided  on  the  Delphi  questionnaire.  The  fact  that 
every  single  panel  member  deemed  this  to  be  a  relevant  requirement  at  some  level  of 
certification  gives  credibility  for  including  it  in  the  ideal  model.  The  research  team 
acknowledges  the  majority  opinion  (64%)  for  including  a  specialty  course  as  a 
requirement  for  the  second  level  of  certification.  Secondly,  the  expert  panel  was  nearly 
split  (55/45)  with  regards  to  contract  law  being  required  at  the  first  or  second  level  of 
certification.  Again,  the  research  team  will  recognize  the  majority  preference  (55%)  of 
the  expert  panel  and  include  Contract  Law  as  a  requirement  for  the  first  level  of 
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certification  under  the  ideal  model.  The  final  course  in  which  the  panel  arrived  at  less 
than  a  decisive  consensus  was  Contract  Process  Improvements.  The  responses  for  this 
course  were  extremely  varied.  In  fact,  this  was  the  only  course  in  which  panel  members 
chose  all  four  available  responses.  This  may  be  attributed  to  the  expert  panel  not  being 
informed  exactly  what  this  course  entailed.  Similar  to  the  specialty  course.  Contract 
Process  Improvements  was  introduced  by  a  panel  member  during  the  interview  process 
and  received  82%  support  for  inclusion  (at  some  level)  in  the  ideal  model.  The  research 
team  will  oblige  the  modal  response  (45%)  of  Level  3,  and  require  Contract  Process 
Improvements  as  a  prerequisite  for  attaining  the  third  level  of  certification  in  the  ideal 
model.  These  course  requirements  will  be  compared  with  the  existing  APDP  and  NCMA 
course  requirements  during  the  stakeholder  assessment  portion  of  this  chapter. 

Question  8  -  Examinations.  Should  certification  imder  the  ideal  contracting 
certification  program  require  an  individual  to  pass  a  comprehensive  examination? 

Response.  55%  Yes,  for  ALL  levels  of  certification 

27%  Yes,  but  only  for  the  most  advanced  certification  level 
18%  No,  an  examination  would  not  be  beneficial 
Analysis.  The  expert  panel  reached  a  majority  consensus  with  regard  to 
implementing  a  comprehensive  examination  requirement  into  the  ideal  certification 
program.  This  response  was  similar  to  that  received  in  the  interview  process  in  which 
only  1 8%  of  the  expert  panel  rejected  the  notion  of  a  comprehensive  examination.  The 
research  team  will  include  a  comprehensive  examination  requirement  for  each  level  of 
certification  under  the  ideal  model.  Aeknowledging  the  inputs  provided  during  the 
interview  process,  the  type  of  test  administered,  the  administration  responsibility,  and  the 
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cost  of  administration  are  all  concerns  that  would  have  to  be  resolved  prior  to  the 
implementation  of  this  requirement.  These  issues  are  beyond  the  scope  of  this  research, 
but  are  addressed  for  reasons  of  practicality.  Despite  these  potential  obstacles,  it  is  clear 
that  a  majority  of  the  expert  panel  (55%)  feel  that  a  comprehensive  examination  would 
enhance  the  ideal  contracting  certification  model.  These  examination  requirement  will  be 
compared  with  the  existing  APDP  and  NCMA  requirements  during  the  stakeholder 
assessment  portion  of  this  chapter. 

Question  9  -  Re-certification.  Under  an  ideal  contracting  certification  program, 
should  periodic  RE-CERTIFICATION  be  required  for  an  individual  to  retain  his/her 
certification? 

Response.  9%  Yes,  via  aperiodic  examination 

73%  Yes,  via  experience  and  continuing  education 
18%  Yes,  via  experience,  education,  and  an  examination 
0%  No 

Analysis.  1 00%  of  the  expert  panel  agreed  that  re-certification  is  an 
important  aspect  of  the  ideal  certification  model.  Although  the  panel  was  somewhat 
divided  over  what  exactly  re-certification  should  entail,  a  strong  consensus  (73%) 
emerged  identifying  contracting  experience  and  continuing  education  as  the  driving 
forces  for  re-certification.  The  research  noted  that  even  though  a  panel  majority  favored 
a  comprehensive  examination  for  initial  certification,  only  27%  supported  a 
comprehensive  exam  for  re-certification.  It  is  assumed  that  the  expert  panel’s  rationale  is 
that  a  certification  exam  will  require  the  demonstration  of  a  sound  knowledge  base  in 
both  regulatory  and  statutory  requirements,  while  further  experience  and  continuing 
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education  will  assist  contracting  personnel  in  keeping  up  with  pertinent  changes  in  the 
field.  Nonetheless,  it  is  clear  that  the  expert  panel  feels  strongly  that  re-certification 
should  be  incorporated  into  the  ideal  model  based  upon  experience  and  continuing 
education.  Since  a  re-certification  process  is  incorporated  into  the  ideal  model,  other 
questions  must  now  be  answered.  How  long  should  certification  be  valid  before  re¬ 
certification  is  required  and  how  much  continuing  education  should  be  required  to  re¬ 
certify?  These  questions  will  now  be  addressed. 

Question  10  -  Validity  Period.  Assuming  a  contracting  certification  program  with 
mandatory  re-certification,  how  long  should  an  individual’s  certification  remain  valid? 

Response.  0%  1  year 

0%  2  years 

27%  3  years 
9%  4  years 
64%  5  years 

Analysis.  Despite  being  divided  among  three,  four,  and  five  years,  the 
panel  reached  a  majority  consensus  (64%)  of  5  years  for  certification  validity.  As  certain 
panel  members  identified  during  the  interview  process,  the  purpose  of  re-certification  is 
to  ensure  that  individuals  stay  abreast  of  continually  changing  contracting  requirements. 
This  five  year  certification  period  is  aimed  at  providing  contracting  personnel  motivation 
to  keep  up  with  changing  requirements,  while  at  the  same  time  not  overburdening  the 
administration  of  the  system  with  frequent  (i.e.,  yearly)  re-certification  processing.  A 
five  year  validity  period  will  be  incorporated  into  the  ideal  model  and  compared  with  the 
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existing  APDP  and  NCMA  requirements  in  the  stakeholder  assessment  portion  of  this 
chapter. 

Question  1 1  -  Continuing  Education.  Assuming  a  contracting  certification 
program  with  mandatory  re-certification,  how  much  continuing  education  (on  average) 
should  be  required  to  maintain  an  individual’s  certification? 

Response.  9%  30  hours  per  year 

73%  40  hours  per  year 

Other  -  please  specify 

9%  16  hours  per  year 
9%  12  hours  per  year 

Analysis.  It  is  important  to  recognize  that  every  panel  member  indicated 
the  need  for  at  least  12  hours  of  annual  continuing  education  for  contracting  personnel. 
While  every  panel  member  indicated  the  need  for  some  continuing  education,  a  strong 
consensus  (73%)  supported  40  classroom  hours  as  the  average  annual  requirement  for  the 
ideal  certification  model.  40  hours  equates  to  one  one-week  course  once  per  year.  The 
research  team  believes  this  figure  to  be  sufficient  for  either  refresher  or  advanced  training 
without  forcing  people  to  miss  significant  periods  of  work  in  order  for  them  to  attend 
various  schools.  As  a  result  of  the  panel’s  strong  consensus,  the  research  team  will 
incorporate  a  continuing  education  requirement  of  40  hours  per  year  into  the  ideal 
contracting  certification  model.  The  complete  certification  model  will  be  presented 
below  followed  by  a  stakeholder  assessment  comparing  the  ideal  model  with  the  current 
APDP  and  NCMA  certification  programs. 
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The  Ideal  Contracting  Certification  Model 


Based  upon  the  interview  process  and  the  Delphi  questionnaire  presented  and 
analyzed  above.  Table  7  represents  the  ideal  contracting  certification  model  as  developed 
by  the  expert  panel  and  research  team. 


Table  7.  The  Ideal  Contracting  Certification  Model 


Requirements 

Ideal  Model 

Levels 

3  levels 

Experience 

Level  1 

2  years 

Level  2 

5  years 

Level  3 

10  years 

Formal  Education 

Level  1 

Bachelor's  Degree 

Level  2 

same  as  above 

Level  3 

same  as  above 

Procurement  Courses 

Level  1 

Contracting  Fundamentals,  Contract  Pricing, 

Contract  Law,  Contract  Negotiation 

Level  2 

Intermediate  Contracting,  Intermediate  Pricing, 
Specialty  Course 

Level  3 

Executive  Contracting,  Advanced  Topics  in 
Acquisition,  Management  of  Contracting  Activities, 
Contract  Process  Improvements 

Comprehensive 

Yes,  all  levels 

Examination 

Re-certification 

Yes,  via  experience  and  continuing  education 

Duration  of  Certification 

5  years 

Continuing  Education 

40  hours  per  year 
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Model  Comparison  and  Stakeholder  Assessment 

The  following  table  compares  the  newly  formulated  ideal  model  with  the  existing 
APDP  and  NCMA  certification  programs. 


Table  8.  Comparison  of  the  Ideal  Model  with  APDP  and  NCMA  Requirements 


Requirements 

Ideal  Model 

APDP 

NCMA 

Levels 

3  levels 

3  levels 

2  levels  (CACM  &  CPCM) 

Experience 

Level  1 

2  years 

1  year 

1  year 

Level  2 

5  years 

2  years 

2  years 

Level  3 

10  years 

4  years 

N/A 

Formal  Education 

Level  1 

Bachelor’s  Degree 

Bachelors  Degree  or  24 
business  related  semester 
hours,  or  10  years 
experience 

1  year  college 

Level  2 

same  as  above 

same  as  above 

Bachelor’s  Degree 

Level  3 

same  as  above 

same  as  above 

N/A 

Procurement  Courses 

Level  1 

Contracting  Fundamentals, 
Contract  Pricing.  Contract 
Law,  Contract  Negotiation 

Contracting  Fundamentals, 
Contract  Pricing 

One  acquisition  course 

Level  2 

Intermediate  Contracting, 
Intermediate  Pricing, 

Specialty  Course 

Intermediate  Contracting, 
Intermediate  Pricing, 

Contract  Law 

Eight  procurement-related  courses 

Level  3 

Executive  Contracting, 
Advanced  Topics  in 
Acquisition,  Management  of 
Contracting  Activities, 
Contract  Process 
Improvements 

Executive  Contracting, 
Management  for  Contract 
Supervisors 

N/A 

Comprehensive  Examination 

Yes,  all  levels 

No 

Yes,  both  levels 

Re-certification 

Yes,  via  experience  and 
continuing  education 

No 

Yes,  via  experience  and  continuing 
education 

Duration  of  Certification 

5  years 

Indefinite 

5  years 

Continuing  Education 

40  hours  per  year 

None 

60  hours  over  5-year  span  - 10  of  60 
in  last  18  months. 

55 


Stakeholder  Assessment.  The  research  team  conducted  a  stakeholder  assessment 
in  the  form  of  a  survey  to  gain  insight  from  an  array  of  personnel  in  the  contracting  field 
as  to  how  the  ideal  model  is  perceived  when  compared  with  the  existing  certification 
programs  offered  by  APDP  and  NCMA.  This  assessment  was  conducted  subsequent  to 
the  development  of  the  ideal  model  and  as  a  result  the  stakeholders  had  no  influence  on 
the  actual  model  development. 

The  stakeholder  assessment  survey  questions  are  listed  in  Appendix  D.  The  survey 
consisted  of  34  questions  relating  to  the  ideal  contracting  certification  model.  The 
questions  were  formulated  by  the  research  team  to  assess  stakeholders’  views  on  all 
elements  of  the  model.  Questions  were  grouped  according  to  requirement  topic 
(experience,  education,  etc.)  and  certification  level.  For  instance,  the  first  question 
surveyed  respondents  on  Level  1  experience  requirements  of  the  Ideal  Model  compared 
to  the  experience  requirements  of  APDP  Level  1.  The  next  question  surveyed 
respondents  on  Level  1  experience  requirements  of  the  Ideal  Model  compared  to  the 
experience  requirements  of  NCMA  Level  1 .  The  research  team  utilized  a  Likert  scale 
with  the  following  choices  to  assess  stakeholders’  opinions:  Much  Worse,  Worse, 
Similar,  Better,  Much  Better.  The  research  team  included  an  open-ended  question  after 
each  related  group  of  questions  to  determine  if  the  respondents  felt  that  the  differences 
between  the  Ideal  Model  and  the  existing  models  were  important.  For  example,  after  the 
first  two  questions  comparing  Level  1  experience  requirements  with  the  experience 
requirements  of  the  existing  levels,  the  third  question  asked  if  the  differences  were 
important  and  why.  This  gave  the  stakeholders  a  chance  to  justify  their  opinions  from  the 
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previous  questions  and  to  share  their  opinions  relating  to  the  differences.  This  pattern 
was  repeated  until  all  requirement  topics  were  compared  to  all  levels  of  both  of  the 
existing  certification  programs. 

Stakeholder  Selection  and  Participation.  The  research  team  wished  to  attain  a 
stakeholder  assessment  through  an  unbiased  and  varied  cross-section  of  personnel  from 
within  the  contracting  career  field  in  terms  of  military/civilian,  officer/enlisted, 
contracting  experience,  and  present  contracting  certifications.  The  research  team 
determined  that  surveys  of  one  hundred  qualified  stakeholders  would  provide  sufficient 
data  to  allow  for  a  valid  assessment.  A  qualified  stakeholder  was  determined  to  be  any 
individual  currently  in  the  contracting  career  field. 

Participation  was  solicited  and  attained  primarily  from  qualified  individuals  attending 
contracting-related  professional  continuing  education  courses  at  the  School  of  Systems 
and  Logistics,  Air  Force  Institute  of  Technology  (AFIT),  Wright-Patterson  Air  Force 
Base,  Ohio.  Other  stakeholders  were  selected  from  among  Graduate  Contract 
Management  students  attending  the  Graduate  School  of  Logistics  and  Acquisition 
Management,  AFIT,  and  Army  contracting  personnel  from  various  locations. 

Results  and  Comments  bv  Question. 

Question  1 .  How  do  the  Level  1  experience  requirements  of  the  proposed 
program  measure  up  to  those  of  the  APDP  program? 


Responses. 

Much  Worse 

4% 

Worse 

25% 

Similar 

27% 

Better 

39% 

Much  Better 

4% 

No  Response 

1% 
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Analysis.  The  responses  indicate  that  stakeholders  view  this  portion  of  the 
model  as  an  improvement  to  the  existing  APDP  requirements  as  43%  indicated  that  two 
years  of  experience  would  he  an  improvement  to  the  existing  APDP  requirement  of  one 
year. 

Question  2.  How  do  the  Level  1  experience  requirements  of  the  proposed 
program  measure  up  to  those  of  the  NCMA  program? 

Responses. 


Much  Worse  4% 

Worse  22% 

Similar  30% 

Better  38% 

Much  Better  4% 

No  Response  2% 


Analysis.  The  responses  indicate  that  stakeholders  view  this  portion  of  the 
model  as  an  improvement  to  the  existing  NCMA  requirements  as  42%  indicated  that  two 
years  of  experience  would  be  an  improvement  to  the  existing  NCMA  requirement  of  one 
year. 

Question  3.  Is  this  difference  important?  Why? 

Specific  Comments.  Some  advocates  of  the  more  stringent  requirements 
believed  that  experience  was  the  most  important  requirement  relating  to  certification. 
Respondents  said  that  even  the  most  basic  contracting  actions  are  becoming  more  and 
more  complex  and  that  additional  experience  would  enhance  the  ability  of  Level  1 
personnel  to  perform  the  mission  effectively.  They  stated  that  one  year  is  “the  ever- 
changing  environment  in  Government  contracting  is  simply  too  complex  these  days  to 
develop  a  significant  level  of  expertise  in  less  than  two  years,”  and  that  “experience 
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counts  a  lot  because  the  real  world  doesn’t  always  follow  along  textbook  guidelines.” 
Another  interesting  observation  was  that  the  “first  year  is  spent  in  school  (procurement 
courses)  as  opposed  to  ‘doing  contracting’.”  Opponents  of  the  more  stringent 
requirements  of  the  ideal  model  believed  that  individuals  “develop  at  different  paces”  and 
that  personnel  should  be  allowed  to  “advance  at  their  own  speed.”  They  stated  that 
longer  experience  requirements  “would  make  everyone  equal  without  allowing  room  for 
the  ‘go  getters’.”  Others  stated  that  “experience  time  minimums  are  not  truly  reflective 
of  what  the  best  people  can  learn.” 

The  research  team  recognizes  that  the  questions  contained  in  this  survey  could  have 
possibly  been  interpreted  differently  among  the  various  stakeholders;  however,  no  such 
evidence  of  multiple  interpretations  was  found  to  exist.  Granted,  the  phrase  “measure 
up”  is  used  in  the  majority  of  questions  throughout  this  survey  and  could  be  interpreted  in 
multiple  ways  when  used  in  an  open-ended  type  question.  However,  the  responses 
available  to  the  respondents  (i.e.,  better,  worse,  similar),  in  these  forced  response  type 
questions,  provided  a  control  mechanism  to  force  respondents  to  compare  the  ideal  model 
with  the  existing  certification  programs  and  identify  whether  they  viewed  it  as  better  or 
worse.  Follow  up  questions,  such  as  Question  3  above,  were  asked  to  provide 
respondents  an  opportunity  to  support  their  decisions  in  the  previous  questions.  The 
research  team  noted  no  discrepancies  between  the  individual  choices  made  to  the  forced 
response  questions  with  the  comments  provided  to  the  open  ended  questions.  In  every 
instance,  the  comments  supported  their  decision.  Based  upon  this  analysis  and  rationale 
the  research  team  is  confident  that  the  stakeholders  held  a  common  understanding  the 
questions  asked  throughout  this  instrument  and  answered  accordingly. 
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Question  4.  How  do  the  Level  2  experience  requirements  of  the  proposed 


program  measure  up  to  those  of  the  APDP  program? 
Responses. 


Much  Worse  9% 

Worse  38% 

Similar  11% 

Better  33% 

Much  Better  5% 

No  Response  4% 


Analysis.  The  responses  indicate  that  stakeholders  do  not  view  this 
portion  of  the  model  to  be  better  than  the  existing  APDP  requirements  as  47%  indicated 
that  five  years  of  contracting  experience  would  in  fact  be  worse  than  the  existing  APDP 
requirement  of  two  years. 

Question  5.  How  do  the  Level  2  experience  requirements  of  the  proposed 
program  measure  up  to  those  of  the  NCMA  program? 

Responses. 


Much  Worse  8% 

Worse  39% 

Similar  15% 

Better  29% 

Much  Better  5% 

No  Response  4% 


Analysis.  The  responses  indicate  that  stakeholders  do  not  view  this 
portion  of  the  model  to  be  better  than  the  existing  NCMA  requirements  as  47%  indicated 
that  five  years  of  contracting  experience  would  in  fact  be  worse  than  the  existing  NCMA 
requirement  of  two  years. 
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Question  6.  Is  this  difference  important?  Why? 

Specific  Comments.  Respondents  opposed  to  this  difference  between  the 
Ideal  Model  and  the  existing  certification  programs  again  pointed  to  the  possibility  that 
such  a  restraint  will  prevent  “fast  burners”  from  escalating  through  the  ranks.  One 
comment  read  “You  will  hold  back  the  high  flyers  who  are  your  future  leaders.”  Other 
opponents  expressed  their  opinions  that  the  contracting  workforce  is  comprised  of  a  large 
group  of  employees  who  are  eligible,  or  almost  eligible  for  retirement.  WTien  they  retire 
it  will  create  a  void  in  Level  2  personnel  and  more  stringent  experience  requirements  will 
prevent  many  personnel  fi-om  achieving  Level  2  status  in  time  to  fill  the  void.  Many 
advocates  of  the  Ideal  Model  experience  requirements  stated  that  the  requirements  of  the 
existing  programs  were  not  high  enough.  One  commented  that,  “Two  years  of 
experience  to  achieve  Level  2  certification  in  a  3-level  program  is  ridiculous.  There’s 
little  distinction  between  entry-level  personnel  and  seasoned  Contracting  Officers.” 
Similarly,  another  stakeholder  said,  “Additional  time  requirement  would  attach  more 
significance  to  Level  2.  Currently,  you  can  receive  a  warrant  w/Level  2  experience. 
Increasing  the  time  to  five  years  would  ensure  a  more  ‘seasoned’  Contracting  Officer.” 
Another  stated,  “How  can  you  say  you  are  a  Certified  Professional  Contracts  Manager 
when  you  have  less  than  five  years  in  the  business?  The  same  thing  goes  for  Level  2 
APDP.  Passing  schools  is  not  enough.  Depending  on  the  area  you  are  in,  even  five  years 
may  not  be  adequate.”  Others  commented  that  “More  experience  in  the  field  will 
produce  a  more  knowledgeable  work  force.” 
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Question  7.  How  do  the  Level  3  experience  requirements  of  the  proposed 
program  measure  up  to  those  of  the  APDP  program? 

Responses. 


Much  Worse  15% 

Worse  33% 

Similar  9% 

Better  31% 

Much  Better  8% 

No  Response  4% 


Analysis.  The  responses  indicate  that  stakeholders  do  not  view  this 
portion  of  the  model  to  be  better  than  the  existing  APDP  requirements  as  48%  indicated 
that  ten  years  of  contracting  experience  would  in  fact  be  worse  than  the  existing  APDP 
requirement  of  four  years. 

Question  8.  Is  this  difference  important?  Why? 

Specific  Comments.  Qpponents  of  these  more  stringent  experience 
requirements  for  Level  3  called  the  requirements  excessive.  Qne  commented,  “A  Captain 
or  GS-1 1/12  should  not  be  deprived  of  a  Level  3  certification  because  they  have  8  or  9 
years  experience.  These  people  may  be  able  to  adequately  serve  as  Flight  Commanders, 
Squadron  Commanders,  or  Division  Chiefs.”  Qther  comments  were  similar  to  those  of 
the  Level  2  requirements,  “The  acquisition  workforce  is  aging.  We  need  to  move 
employees  to  Level  3  as  quickly  as  possible.  Ten  years  experience  is  too  long.” 
Advocates  of  the  Ideal  Model  experience  requirements  felt  that  four  years  was  simply  not 
enough  time  to  achieve  the  highest  level  of  certification  in  a  career  field.  Qne 
commented,  “A  person  having  four  years  experience,  a  Bachelors  and  some  additional 
classes  can  be  certified  at  the  highest  level  possible  in  contracting.  I  consider  the  ‘highest 
level  possible’  as  a  benchmark.  Is  this  the  benchmark  the  Air  Force  wants  to  maintain?” 
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Question  9.  How  do  the  Level  1  education  requirements  of  the  proposed  program 


measure  up  to  those  of  the  APDP  program? 
Responses. 


Much  Worse  8% 

Worse  32% 

Similar  33% 

Better  18% 

Much  Better  7% 

No  Response  2% 


Analysis.  The  responses  indicate  that  stakeholders  do  not  view  this 
portion  of  the  model  to  be  better  than  the  existing  APDP  requirements  as  40%  of  the 
respondents  indicated  that  the  requirement  for  a  bachelor’s  degree  is  worse  or  much 
worse  than  the  APDP  requirement  for  either  a  bachelor’s  degree,  24  business  related 
semester  hours,  or  10  years  of  contracting  experience. 

Question  10.  How  do  the  Level  1  education  requirements  of  the  proposed 
program  measure  up  to  those  of  the  NCMA  program? 

Responses. 


Much  Worse  6% 

Worse  23% 

Similar  17% 

Better  32% 

Much  Better  19% 

No  Response  3% 


Analysis.  The  responses  indicate  that  stakeholders  view  this  portion  of  the 
model  as  an  improvement  to  the  existing  NCMA  requirements  as  5 1%  of  the  respondents 
indicated  that  the  requirement  for  a  bachelor’s  degree  is  better  or  much  better  than  the 
NCMA  requirement  of  one  year  of  college. 
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Question  1 1 .  Is  this  difference  important?  Why? 

Specific  Comments.  Advocates  of  these  differences  stressed  that  a 
bachelor’s  degree  is  critical  to  entering  the  contracting  career  field.  One  stated,  “No  one 
should  enter  the  contracting  field  in  today’s  business  environment  with  less  than  a 
bachelor’s  degree.”  Others  felt  the  requirement  of  a  bachelor’s  degree  without  the 
business  credits  opened  up  the  career  field.  One  comment  read,  “Anyone  with  a  college 
degree  could  get  their  foot  in  the  door,  not  just  business  majors  or  minors.”  Another 
stated,  “Some  of  our  best  acquisition  professionals  are  liberal  arts  graduates.  They  come 
with  an  open  mind  and  are  very  trainable.  As  they  progress,  the  attainment  of  business 
education  is  more  important  and  meaningful.  It  should  not  be  a  prerequisite  for  the  more 
basic  certification  level.  Conversely,  one  response  stated  that  “A  bachelor’s  should  not 
be  required  for  Level  1 .  This  would  exclude  many  qualified  personnel  with  less 
education.”  In  a  comparison  to  the  NCMA  requirements,  another  felt  that  “a  bachelor’s 
degree  should  be  required  for  CACM  also.”  Those  stakeholders  opposed  to  the  different 
requirements  of  the  Ideal  Model  felt  the  business  related  credits  of  the  APDP  program 
were  vital.  One  stakeholder  commented,  “A  bachelor’s  degree  in  history/political 
science/zoology,  etc.  has  little  or  no  bearing  on  contracting.”  A  similar  comment  stated, 
“Having  a  degree  with  no  specific  requirement  for  business  courses  is  like  not  requiring  a 
degree.  You  need  the  business/management  courses.” 
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Question  12.  How  do  the  Level  2  education  requirements  of  the  proposed 


program  measure  up  to  those  of  the  APDP  program? 
Responses. 


Much  Worse  5% 

Worse  22% 

Similar  52% 

Better  14% 

Much  Better  5% 

No  Response  2% 


Analysis.  The  education  requirements  for  Level  2  for  both  the  Ideal 
Model  and  the  APDP  program  are  identical  to  those  requirements  for  Level  1 .  However, 
instead  of  viewing  the  requirements  worse  or  much  worse  as  they  did  for  the  Level  1 
evaluation,  52%  of  the  respondents  viewed  this  requirement  as  similar  or  equal. 

Question  13.  How  do  the  Level  2  education  requirements  of  the  proposed 
program  measure  up  to  those  of  the  NCMA  program? 

Responses. 


Much  Worse  4% 

Worse  10% 

Similar  68% 

Better  10% 

Much  Better  4% 

No  Response  4% 


Analysis.  The  Level  2  education  requirements  under  the  Ideal  Model  and 
the  NCMA  program  are  identical.  Therefore,  68%  of  respondents  viewed  the 
requirements  as  being  similar. 

Question  14.  Is  this  difference  important?  Why? 

Specific  Comments.  Comments  were  similar  in  nature  to  those  from 
Question  14.  This  is  likely  due  to  the  fact  that  the  requirements  are  similar. 
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Question  15.  How  do  the  Level  3  education  requirements  of  the  proposed 
program  measure  up  to  those  of  the  APDP  program? 

Responses. 


Much  Worse  9% 

Worse  25% 

Similar  51% 

Better  8% 

Much  Better  4% 

No  Response  3% 


Analysis.  The  education  requirements  for  Level  3  of  both  the  Ideal  Model 
and  the  APDP  program  are  identical  to  those  requirements  for  Levels  1  and  2.  However, 
instead  of  viewing  the  requirements  worse  or  much  worse  as  they  did  for  the  Level  1 
evaluation,  51%  of  the  respondents  viewed  this  requirement  as  similar  or  equal. 

Question  16.  Is  this  difference  important?  Why? 

Specific  Comments.  Again,  because  the  requirements  are  similar  to  those 
for  Levels  1  and  2,  comments  were  very  similar  in  nature  to  those  from  Questions  1 1  and 
14.  However,  several  comments  were  made  that  stated  the  education  requirement  for  the 
Ideal  Model  should  contain  business-related  studies.  Those  same  respondents  suggested 
that  Level  3  require  a  master’s  degree  in  Procurement/Contracting. 

Question  17.  How  do  the  Level  1  course  requirements  of  the  proposed 
program  measure  up  to  those  of  the  APDP  program? 


Responses. 

Much  Worse 

5% 

Worse 

25% 

Similar 

19% 

Better 

39% 

Much  Better 

9% 

No  Response 

3% 
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Analysis.  48%  of  respondents  viewed  procurement  courses  under  the 


ideal  model  as  an  improvement  to  the  required  courses  under  the  APDP  program.  Thus, 
they  believe  that  Contract  Law  and  Contract  Negotiation  classes  should  be  required  for 
Level  1  certification  in  addition  to  Contracting  Fundamentals  and  Contract  Pricing. 

Question  18.  How  do  the  Level  1  course  requirements  of  the  proposed 
program  measure  up  to  those  of  the  NCMA  program? 

Responses. 


Much  Worse  2% 

Worse  19% 

Similar  8% 

Better .  40% 

Much  Better  28% 

No  Response  3% 


Analysis.  The  responses  indicate  that  stakeholders  view  this  portion  of  the 
model  as  an  improvement  to  the  existing  NCMA  procurement  course  requirements  as 
68%  indicated  that  the  requirement  for  four  classes  under  the  Ideal  Model  is  better  or 
much  better  than  the  requirement  for  one  acquisition  course  under  the  NCMA  program. 

Question  19.  Is  this  difference  important?  Why? 

Specific  Comments.  Advocates  of  the  procurement  course  requirements 
for  the  Ideal  Model  believed  it  to  be  important  that  all  of  the  basic  courses  were 
completed  prior  to  any  certifications.  As  one  respondent  stated,  “I  believe  it  is  important 
to  have  the  fundamentals  out  of  the  way  prior  to  achieving  any  Level  1  certification.” 
Another  stated,  “The  more  courses  taken  at  earlier  stages  in  one’s  career,  the  better  the 
individual  will  perform.”  Several  conveyed  that  the  courses  required  under  the  Ideal 
Model  are  the  right  mix  of  courses  needed.  Qne  respondent  stated  “The  courses  proposed 
are  the  right  ones  to  start  and  doable  within  the  first  year”  while  another  stated  “The 
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proposed  courses  appear  to  be  an  appropriate  minimum  for  ‘professional’  certification.” 
Several  comments  were  course  specific:  “Great  idea!  Law  and  negotiations  should  be 
taught  at  this  fundamental  level.”  “Important  to  give  newcomers  a  negotiation  course. 
Less  important  to  give  them  a  law  course.”  “The  law  course  is  probably  the  most 
important  one  we  get.”  Opponents  felt  that  current  requirements  were  stringent  enough. 
One  stated  that  “Adding  the  two  additional  courses  puts  a  great  deal  of  stress  on  a  trainee 
who  probably  is  not  using  those  skills  yet.” 

Question  20.  How  do  the  Level  2  course  requirements  of  the  proposed 
program  measure  up  to  those  of  the  APDP  program? 

Responses. 


Much  Worse  1% 

Worse  10% 

Similar  65% 

Better  20% 

Much  Better  3% 

No  Response  1% 


Analysis.  The  results  indicate  that  most  stakeholders  (68%)  view  the 
procurement  course  requirements  between  the  Ideal  Model  and  APDP  very  similar. 

Question  21.  How  do  the  Level  2  course  requirements  of  the  proposed 
program  measure  up  to  those  of  the  NCMA  program? 


Responses. 

Much  Worse 

2% 

Worse 

12% 

Similar 

42% 

Better 

26% 

Much  Better 

10% 

No  Response 

8% 
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Analysis.  The  results  indicate  that  the  largest  portion  of  stakeholders 
(42%)  view  the  procurement  course  requirements  between  the  Ideal  Model  and  NCMA 
very  similar  even  though  the  NCMA  program  at  this  level  requires  eight  procurement- 
related  courses. 

Question  22.  Is  this  difference  important?  Why? 

Specific  Comments.  Comments  received  were  similar  to  those  in 
Question  1 9.  However,  with  regard  to  the  NCMA  courses,  many  respondents  were 
hesitant  to  express  an  opinion  without  knowing  what  specific  courses  are  required.  As 
one  respondent  put  it,  “May  be  better  or  worse  -  depends  on  subjects  and  quality.” 

Question  23.  How  do  the  Level  3  course  requirements  of  the  proposed  program 
measure  up  to  those  of  the  APDP  program? 

Responses. 


Much  Worse  0% 

Worse  21% 

Similar  14% 

Better  46% 

Much  Better  14% 

No  Response  5% 


Analysis.  The  responses  indicate  that  stakeholders  view  this  portion  of  the 
model  as  an  improvement  to  the  existing  APDP  Level  3  procurement  course  requirements 
as  60%  of  the  respondents  indicated  that  the  requirement  for  the  four  classes  imder  the 
Ideal  Model  is  better  or  much  better  than  the  two  courses  required  under  the  APDP 
program. 

Question  24.  Is  this  difference  important?  Why? 

Specific  Comments.  Many  comments  were  simply  of  the  opinion  that 
more  is  better  at  every  level  of  certification  when  it  comes  to  education.  More  specific 
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comments  were  “Continuing  education  provides  assurance  that  you  remain  current  and 
grow”  and  “At  Level  3,  contracting  personnel  need  more  course  work  to  focus  them  on 
the  fact  that  they  are  in  the  positions  which  must  be  strategic  and  forward-thinking.”  An 
opponent  stated,  “At  this  point  in  contracting,  you  should  be  attending  seminars  and 
professional  development  voluntarily.  DoD  ‘force  feeding’  reaches  a  point  of 
diminishing  returns  with  two  additional  courses.” 

Question  25.  Is  the  requirement  to  pass  a  comprehensive  examination  an 
enhancement  relative  to  the  existing  APDP  program? 

Responses. 


Much  Worse  14% 

Worse  26% 

Similar  11% 

Better  34% 

Much  Better  8% 

No  Response  7% 


Analysis.  The  largest  portion  of  respondents  (42%)  indicated  that  the 
requirement  to  pass  a  comprehensive  examination  is  better  than  the  lack  of  such  a 
requirement  under  the  APDP  program.  However,  almost  as  many  respondents  (40%) 
evaluated  this  requirement  as  worse  or  much  worse. 

Question  26.  Is  this  difference  important?  Why? 

Specific  Comments.  Advocates  of  the  requirement  for  a  comprehensive 
examination  felt  that  such  a  requirement  would  ensure  a  knowledgeable  workforce.  Qne 
response  commented  that  “This  would  provide  a  ‘standard’  baseline’  of  knowledge  for  all 
to  meet.  As  a  profession,  acquisition  should  confine  entry  to  the  most  qualified  people.” 
Another  commented  that  An  examination  demonstrates  proficiency.  If  an  individual  is 
not  proficient,  no  level  of  certification  should  be  issued  to  that  individual.”  Another 


comment  addressed  knowledge  comprehension,  stating  “It  (a  comprehensive 
examination)  illustrates  that  the  knowledge  has  been  integrated  and  not  merely  duplicated 
for  a  test.”  Many  opponents  of  this  requirement  commented  on  the  possibility  that  some 
individuals  are  simply  not  proficient  at  taking  test.  One  commented  that  “You  could 
have  one  of  the  most  experienced,  knowledgeable  contracting  individuals  working  with 
you,  but  they  may  not  test  well.”  Other  comments  made  doubted  that  a  “fair”  test  could 
be  developed,  such  as  “The  contracting  field  is  so  broad  that  I  don’t  know  how  you 
would  devise  a  ‘fair’  exam  -  many  of  us  work  in  Post,  Camp  and  Station  and  have  to  deal 
with  parts  of  the  FAR  that  are  obviously  written  by  and  for  the  systems  folks.” 

Question  27.  Is  the  proposed  re-certification  requirement  an  enhancement 
relative  to  the  existing  APDP  program? 

Responses. 


Much  Worse  10% 

Worse  20% 

Similar  13% 

Better  43% 

Much  Better  11% 

No  Response  3% 


Analysis.  The  responses  indicate  that  most  stakeholders  believe  the 
requirement  for  re-certification  under  the  Ideal  Model  is  an  improvement  to  the  lack  of 
such  re-certification  requirements  under  the  APDP  program  as  54%  indicated  this 
requirement  to  be  better  or  much  better. 

Question  28.  Is  this  difference  important?  Why? 

Specific  Comments.  Advocates  focused  on  the  need  for  the  workforce  to 
stay  current.  One  commented  that  “Periodic  reviews  would  keep  skills  sharp”  while 
another  stated  “re-certification  maintains  competency  of  the  workforce.”  Others  pointed 


71 


to  the  requirement  for  re-certification  in  other  fields,  “Professional  eareer  fields  all 
require  ongoing  certification  -  so  should  contracting.”  Opponents  felt  that  re¬ 
certification  requirements  would  present  too  much  of  a  burden  to  personnel.  One 
commented,  “It  is  already  difficult  to  do  my  job  and  keep  up  with  changing  requirements. 
I  don’t  want  to  worry  about  re-certifying  every  five  years.” 

Question  29.  How  does  the  proposed  duration  of  certification  measure  up  to  that 
of  the  existing  APDP  program? 

Responses. 


Much  Worse  11% 

Worse  30% 

Similar  10% 

Better  36% 

Much  Better  9% 

No  Response  4% 


Analysis.  The  responses  indicate  that  the  largest  portion  of  stakeholders 
(45%)  view  a  certification  length  of  five  years  under  the  Ideal  Model  as  better  or  much 
better  than  the  indefinite  certification  under  the  APDP  program.  However,  almost  as 
many  respondents  (41%)  rated  this  requirement  worse  or  much  worse. 

Question  30.  How  does  the  proposed  duration  of  certification  measure  up  to  that 
of  the  existing  NCMA  program? 

Responses. 


Much  Worse  4% 

Worse  9% 

Similar  76% 

Better  6% 

Much  Better  1% 

No  Response  4% 
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Analysis.  The  requirements  under  the  Ideal  Model  and  the  NCMA 


program  are  identical,  explaining  the  fact  that  76%  of  respondents  viewed  the 
requirements  similar. 

Question  31.  Is  this  difference  important?  Why? 

Specific  Comments.  Advocates  of  certification  duration  requirements  for 
the  Ideal  Model  point  out  the  dynamic  environment  of  contracting.  One  commented, 
“Certification  on  a  smaller  scale  would  be  better  every  three  years,  especially  with  the 
number  of  reforms  and  law  changes.  This  is  not  a  static  career  field.”  Others  point  out 
that  such  a  requirement  forces  personnel  to  train,  “If  you  go  for  a  longer  period,  people 
tend  to  put  off  until  next  year  the  training  they  need  today.”  Another  advocate 
commented,  “A  lot  has  changed  in  recent  years  -  to  that  someone  certified  five  or  ten 
years  ago  should  still  be  certified  is  risky.”  Another  stated,  “If  re-certification  is 
dependent  upon  passing  an  exam,  it  would  verify  skills.  If  not,  it  would  at  least  provide 
for  a  periodic  review  of  personnel  efficiency.”  Opponents  simply  saw  no  need  for  re¬ 
certification  as  one  commented,  “Certification  should  stay  indefinite  with  continuing 
education  and  performance  as  a  basis.” 

Question  32.  How  do  the  proposed  continuing  education  requirements  measure 
up  to  that  of  the  existing  APDP  program? 

Responses. 


Much  Worse  5% 

Worse  20% 

Similar  8% 

Better  46% 

Much  Better  16% 

No  Response  5% 
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Analysis.  The  results  indicate  that  most  (62%)  of  the  stakeholders  view 


the  continuing  education  requirements  of  40  hours  per  year  imder  the  Ideal  Model  as 
better  or  much  better  than  the  lack  of  such  requirements  under  the  APDP  program. 

Question  33.  How  do  the  proposed  continuing  education  requirements  measure 
up  to  that  of  the  existing  NCMA  program? 

Responses. 


Much  Worse  2% 

Worse  24% 

Similar  20% 

Better  40% 

Much  Better  8% 

No  Response  6% 


Analysis.  The  results  indicate  that  the  largest  portion  (48%)  of 
stakeholders  view  the  continuing  education  requirements  of  40  hours  per  year  under  the 
Ideal  Model  as  better  or  much  better  than  the  less  stringent  continuing  education 
requirements  vmder  the  NCMA  program. 

Question  34.  Is  this  difference  important?  Why? 

Specific  Comments.  Advocates  of  the  Ideal  Model’s  continuing  education 
requirements  made  comments  similar  to  those  in  the  re-certification  question.  Qne  stated, 
“Continuing  education  require  folks  to  learn  new  things,  stay  current  and  not  stagnate.” 
Another  commented,  “It  will  force  people  to  leave  their  desk  and  look  outside  the  box.” 
Qpponents  of  this  requirement  did  not  aim  comments  at  continuing  education,  rather  at 
the  amount  of  continuing  education  proposed  by  the  Ideal  Model.  Qne  referred  to  40 
hours  annually  as  “unrealistic.”  Another  suggested  that  “40  hours  per  year  is  too  much 
unless  there  is  some  way  to  put  a  course  on  the  internet  each  year  and  make  it  easy  for 
employees  to  obtain  this  education  during  normal  duty  hours.” 


74 


V.  Conclusions  and  Recommendations 


Introduction 

This  chapter  presents  the  research  team’s  findings  and  conclusions  to  the  five  research 
question  posed  in  this  study.  Subsequently,  the  team  presents  its  suggestions  for  further 
research. 

Conclusions 

Research  Question  1 .  How  many  levels  of  certification  should  an  ideal  contracting 
certification  program  contain? 

The  expert  panel  strongly  supported  multiple  levels  of  certification  even  though  a 
small  minority  suggested  that  multiple  levels  might  degrade  the  prestige  of  certification. 
The  panel  expressed  the  opinion  that  multiple  levels  of  certification  could  effectively  be 
utilized  for  recognition  and  motivation  purposes.  Expert  panel  members  concluded  by  a 
narrow  margin  that  three  levels  of  certification  should  be  included  in  an  ideal  contracting 
certification  program. 

The  results  indicate  that  either  of  the  current  certification  programs  would  find  their 
proponents  based  on  the  definition  of  the  ideal  system.  The  current  APDP  certification 
program  provides  a  sufficient  number  of  levels  to  capture  the  requirements  for 
progression  in  the  contracting  profession.  However,  the  current  NCMA  certification 
program  may  not  be  sufficient  in  this  respect  as  it  maintains  only  two  levels  of 
professional  certification,  the  CACM  and  CPCM. 
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Research  Question  2.  What  should  be  the  education  and  experience  requirements  for 
contracting  personnel  to  attain  certification? 

Input  received  early  in  the  study  showed  a  large  variation  with  regard  to  the  number  of 
years  of  experience  that  should  be  required.  Information  was  collected  on  both  a  two- 
level  and  a  three-level  certification  program.  The  first  research  question  was  answered 
with  the  determination  for  a  three-level  system.  Therefore,  only  the  panel’s  input  for  a 
three-level  system  will  be  discussed  here. 

For  the  first  level  of  certification  the  panel  concluded  by  a  majority  that  two  years  of 
contracting  experience  would  suffice.  Stakeholders  strongly  agreed  with  this  assessment 
suggesting  that  experience  requirements  under  the  existing  APDP  and  NCMA  programs 
may  be  deficient  and  stressing  the  importance  of  experience  itself 

The  second  level  of  certification  was  not  as  strongly  supported  by  either  the  panel  or 
the  stakeholders.  The  panel  soundly  concluded  that  between  four  and  six  years 
experience  should  be  required  for  Level  2  certification.  The  most  common  response  was 
for  five  years  with  only  slightly  less  support  for  four  and  six  years  of  experience.  All  of 
these  positions  suggest  more  stringent  experience  requirements  for  the  second  level  of 
certification  than  required  by  the  existing  programs. 

The  majority  of  stakeholders  disagreed  with  the  expert  panel  on  this  position. 
Stakeholder  comments  suggested  that  the  reason  for  such  disagreement  might  be  fear  of 
slowing  career  progression.  Several  stakeholders  believed  that  this  requirement  might 
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hold  back  “fast  burners”  in  the  career  field.  These  results  may  be  due  to  the  difference  in 
relative  experience  between  the  two  groups. 

Results  for  the  third  level  of  certification  were  much  more  decisive  from  the  expert 
panel  as  they  overwhelmingly  concluded  the  requirement  for  ten  years  of  experience 
prior  to  certification  at  the  highest  level.  Again,  the  largest  portion  of  stakeholders 
disagreed  with  the  views  of  the  panel  expressing  concern  over  such  stringent 
requirements  with  respect  to  career  progression.  Regardless  of  the  dissenting  views 
expressed  by  the  stakeholders,  the  ten  years  of  experience  determined  necessary  imder 
the  Ideal  Model  conveys  the  importance  of  the  expert’s  belief  that  a  significant  amount  of 
experience  is  required  at  the  highest  level  of  certification.  The  requirement  is  far  in 
excess  of  the  experience  requirements  under  the  two  existing  certification  programs  and 
demonstrates  the  need  for  higher  levels  of  experience  prior  to  certification  at  the  highest 
level. 

While  the  expert  panel  was  initially  divided  over  the  importance  of  formal  education 
with  respect  to  certification,  subsequent  rounds  of  questioning  revealed  support  for  this 
aspect  of  certification.  The  panel’s  determination  of  requiring  a  bachelor’s  degree  for 
every  level  of  certification  was  supported  by  stakeholders.  However,  many  stakeholders 
expressed  that  a  business-related  education  was  still  critical  in  the  contracting  career 
field.  The  requirement  for  a  bachelor’s  degree  at  the  entry  and  subsequent  levels  of 
certification  is  a  necessary  requirement.  As  expressed  by  one  panel  member,  it 
demonstrates  the  ability  to  learn.  The  requirement  is  very  similar  to  the  existing  APDP 
requirement  in  this  area.  However,  the  requirement  may  demonstrate  a  potential 
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shortcoming  in  the  existing  NCMA  program  as  there  is  no  requirement  for  a  degree  of 
any  kind  for  entry-level  certification  within  NCMA. 

Research  Question  3.  Should  the  successful  completion  of  a  comprehensive  examination 
be  required  for  contracting  personnel  to  attain  certification? 

The  expert  panel  reached  a  strong  consensus  during  the  interview  process  that  a 
comprehensive  examination  should  be  required  under  the  ideal  certification  program. 

The  interview  responses  were  further  refined  through  the  Delphi  phase  of  research  in 
which  a  majority  (55%)  of  panel  members  identified  that  such  an  examination  should  be 
required  for  each  level  of  certification.  A  small  number  of  panel  members  (27%) 
supported  a  comprehensive  examination,  but  only  for  the  highest  level  of  certification. 

These  results  indicate  that  the  current  APDP  certification  program  could  possibly  be 
improved  by  implementing  comprehensive  examinations  for  each  certification  level. 
Currently,  no  comprehensive  examinations  are  required  at  any  phase  of  the  APDP 
program;  however,  individuals  are  required  to  pass  curriculum  specific  examinations 
while  attending  required  contracting  courses. 

The  current  NCMA  certification  program  matches  the  ideal  model  with  regards  to  the 
comprehensive  examination  requirements.  Under  the  NCMA  program,  comprehensive 
testing  is  required  at  both  levels  of  certification.  The  CACM  requires  the  completion  of 
an  in-depth  multiple  choice  examination,  while  the  CPCM  requires  the  completion  of  a 
six-hour  essay  response  examination.  The  expert  panel  strongly  felt  that  a 
comprehensive  examination  is  a  vital  part  of  the  ideal  contracting  certification  model,  as 
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it  requires  individuals  to  demonstrate  their  overall  contracting  knowledge  prior  to  being 
granted  a  contracting  certification. 

The  responses  to  the  stakeholder  assessment  with  regards  to  the  comprehensive 
examination  requirement  were  evenly  divided.  42%  of  the  stakeholders  surveyed 
indicated  that  the  inclusion  of  such  a  requirement  in  the  ideal  model  is  an  improvement 
over  the  existing  APDP  program,  while  40%  indicated  that  this  requirement  makes  the 
ideal  model  “worse”  or  “much  worse”  than  the  existing  APDP  program.  Based  upon  the 
comments  provided  in  response  to  this  question,  the  research  team  perceives  that 
stakeholders  who  are  confident  in  their  contracting  ability,  or  who  supervise  other 
contracting  personnel,  tend  to  favor  the  idea  of  the  comprehensive  examination.  The 
rationale  is  that  ideally  this  requirement  would  enhance  the  overall  knowledge  level  of 
the  contracting  field.  On  the  other  hand,  personnel  just  entering  the  contracting  field  or 
personnel  only  concerned  with  the  specifics  of  their  present  position,  tend  to  view  this 
requirement  as  an  inconvenience  and  possibly  feel  threatened  by  being  called  upon  to 
demonstrate  a  broad  knowledge  base  of  Government  contracting. 

Research  Question  4.  Should  certified  contracting  personnel  be  subject  to  re-certification 
throughout  their  contracting  career? 

The  expert  panel  reached  a  strong  (70%)  consensus  during  the  interview  process  that 
periodic  re-certification  should  be  required  under  the  ideal  certification  program.  The 
interview  responses  were  further  refined  through  the  Delphi  phase  of  research  in  which 
the  majority  (73%)  of  panel  members  identified  that  re-certification  should  be  required 
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based  upon  contracting  experience  and  continuing  education.  9%  of  the  panel  supported 
re-certification  based  upon  periodic  examinations  while  1 8%  felt  that  re-certification 
should  be  based  upon  an  examination  as  well  as  additional  experience  and  continuing 
education.  100%  of  the  panel  agreed  during  the  Delphi  phase  that  re-certification  should 
be  required  in  some  form  or  another. 

These  results  indicate  that  the  current  APDP  certification  program  could  possibly  be 
improved  by  implementing  re-certification  requirements.  Currently,  there  are  no 
requirements  for  re-certification  under  the  APDP  program.  Several  comments  by  the 
expert  panel  indicated  that  they  favored  a  re-certification  process,  simply  to  encourage 
personnel  to  keep  of  with  the  continually  changing  procurement  laws  and  regulations. 

The  panel  felt  that  this  could  be  accomplished  through  additional  experience  and 
continuing  education. 

The  current  NCMA  certification  program  is  very  similar  to  the  ideal  model  with 
regards  to  re-certification  requirements.  Under  the  NCMA  program,  re-certification  is 
required  for  both  the  CACM  and  CPCM  and  is  also  based  on  experience  and  continuing 
education.  Both  of  the  NCMA  certifications  are  valid  for  5  years,  and  continuing 
education  requirements  of  60  hours  over  this  five  year  period,  with  10  hours  being 
completed  within  the  last  18  months  are  required  for  re-certification. 

Under  the  ideal  model,  the  expert  panel  also  adopted  a  five-year  re-certification 
period;  however,  the  continuing  education  requirements  are  more  demanding  that  those 
required  by  NCMA.  The  expert  panel  reached  a  strong  consensus  (73%)  that  an  average 
of  40  hours  per  year  of  continuing  education  should  be  required  for  re-certification.  This 
is  equivalent  to  one,  one-week  course  per  year.  The  expert  panel  felt  that  this  continuing 
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education  requirement  will  assist  individuals  with  keeping  abreast  of  changes  in  the 
contracting  field,  while  not  keeping  them  out  of  the  office  for  prolonged  periods  of  time. 

The  responses  from  the  stakeholder  assessment  supported  the  implementation  of  re¬ 
certification  requirements.  A  majority  (54%)  of  the  stakeholders  surveyed  indicated  that 
the  inclusion  of  such  a  requirement  in  the  ideal  model  is  an  improvement  over  the 
existing  APDP  program,  while  just  30%  indicated  that  this  requirement  makes  the  ideal 
model  “worse”  or  “much  worse”  than  the  existing  APDP  program.  In  addition,  when 
asked  to  compare  the  ideal  model’s  five-year  re-certification  period  with  the  indefinite 
certification  period  of  the  current  APDP  program,  45%  favored  the  five-year  period, 
white  4 1  %  preferred  the  indefinite  certification  period.  Although  not  a  majority 
consensus,  the  research  team  concludes  that  more  stakeholders  prefer  a  five-year  re¬ 
certification  period  over  the  existing  APDP  system.  With  regards  to  the  amount  of 
continuous  education  required  for  re-certification,  62%  of  the  stakeholders  identified  the 
40-hour  per  year  requirement  of  the  ideal  model  over  the  non-existent  requirement  of 
APDP.  In  addition,  48%  of  the  stakeholders  preferred  the  40-hour  per  year  requirement 
over  the  60-hour/5-year  requirement  of  NCMA.  26%  preferred  the  NCMA  requirements 
for  continuing  education. 

Based  upon  the  comments  provided  in  response  to  this  question,  the  research  team 
perceives  that  the  majority  of  stakeholders  want  to  keep  abreast  of  changes  in  the 
contracting  field,  and  want  to  be  afforded  the  opportunity  to  attend  contracting  courses 
even  after  they  attain  a  certain  certification  level.  The  research  team  perceives  that  some 
stakeholders  may  have  been  rushed  through  classes,  granted  certification,  and  then  not 
afforded  the  opportunity  to  attend  any  future  classes,  because  they  are  already  certified. 
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The  research  team  recognizes  that  funding,  training  allocations,  and  mission 
requirements,  often  dictate  when  personnel  can  attend  training  courses;  however,  both  the 
expert  panel  and  the  stakeholders  seem  to  agree  on  the  importance  of  continuing 
education  and  re-certification. 

Research  Question  5.  What  aspects  of  the  ideal  certification  program  are  unique  to 
existing  contracting  certification  programs? 

As  identified  in  Table  8,  Comparison  of  the  Ideal  Model  with  APDP  and  NCMA,  the 
ideal  model  is  basically  a  hybrid  of  the  two  existing  contracting  certification  programs. 
The  research  team,  through  the  use  of  the  expert  panel,  did  not  identify  any  truly  unique 
characteristics  for  the  ideal  model  that  were  not  already  implemented  in  either  the  APDP 
or  NCMA  programs. 

The  research  team  attempted  to  gain  a  unique  and  innovative  perspective  fi-om  the 
expert  panel  during  the  interview  process  by  asking  open-ended  interview  style  questions. 
The  research  team  felt  that  by  conducting  interviews  prior  to  formulating  the  Delphi 
questionnaire,  the  expert  panel  would  be  free  to  express  their  own  ideas  without  being 
guided  or  limited  in  their  responses.  Unfortunately,  based  upon  the  interview  responses 
the  expert  panel  appeared  to  have  pre-conceived  notions  based  upon  the  existing 
certification  programs. 

As  a  result,  the  ideal  model  does  not  contain  any  elements  that  are  truly  unique  to  both 
existing  contracting  certification  programs.  While  certain  elements  of  the  ideal  model 
may  be  unique  to  an  existing  program  (i.e.,  the  comprehensive  examination  requirement 
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when  compared  with  APDP),  no  elements  are  unique  to  all  programs  (i.e.,  NCMA 
already  requires  comprehensive  examinations). 

While  the  ideal  model  does  not  introduce  any  truly  unique  elements  to  the  certification 
process,  the  research  team  feels  that  it  does  contain  the  best  elements  of  each  of  the 
existing  programs.  This  is  supported  by  both  the  expert  panel  and  the  stakeholder’s 
assessment. 

Recommendations 

Based  upon  the  input  provided  by  the  expert  panel,  it  would  appear  that  the  two 
existing  certification  programs  combined  (APDP  and  NCMA)  adequately  represent  all 
the  elements  of  an  ideal  contracting  certification  program.  However,  individually, 
neither  of  the  existing  programs  are  necessarily  ideal  in  themselves.  For  example, 

NCMA  requires  comprehensive  examinations  and  re-certification,  but  is  only  comprised 
of  two  certification  levels.  In  contrast,  APDP  is  comprised  of  three  certification  levels, 
but  does  not  incorporate  comprehensive  examinations  or  re-certification  requirements. 

The  research  team  believes,  based  upon  the  research  conducted,  that  the  NCMA 
program  is  closer  to  matching  the  ideal  model,  than  the  APDP  system.  However,  both 
programs  could  make  improvements  towards  matching  the  ideal  model.  The  following 
recommendations  are  provided  for  each  of  the  two  existing  programs. 

Levels  of  Certification.  Based  upon  input  provided  by  the  expert  panel,  it  would 
seem  that  three  levels  of  certification  are  considered  to  be  ideal.  APDP  currently  offers 
three  levels  of  certification,  and  thus  it  is  recommended  that  they  continue  to  offer  three 
levels.  NCMA  currently  offers  two  levels  of  certification;  however,  they  have  recently 
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implemented  a  Simplified  Acquisition  Specialist  credential  that  is  not  necessarily  a 
certification  level,  but  is  a  credential  that  can  be  achieved  by  personnel  not  yet  eligible  or 
willing  to  certify  as  a  CACM  or  CPCM.  The  research  team  recommends  that  this 
Simplified  Acquisition  credential  be  converted  to  an  entry-level  certification  by 
expanding  the  knowledge  base  beyond  simplified  acquisition  to  encompass  a  slightly 
broader  contracting  knowledge  base.  This  modification  would  provide  NCMA  with  the 
three  certification  levels  determined  to  be  ideal  by  the  expert  panel. 

Experience.  It  is  recommended  that  the  experience  required  to  attain  each 
certification  level  under  APDP  or  the  NCMA  program  be  increased  and  spread  out  more 
over  an  individual’s  contracting  career.  Under  APDP,  an  individual  is  eligible  for  the 
highest  level  of  certification  after  only  four  years  of  contracting  experience  (two  years 
experience  for  the  highest  NCMA  certification).  The  research  team  concludes,  based 
upon  the  input  received  by  the  expert  panel,  that  four  years  of  experience  is  not  enough 
time  to  be  considered  an  expert  in  the  field,  and  thus  should  not  suffice  to  achieve  the 
highest  level  of  certification.  While  the  results  of  the  stakeholder  assessment  indicated 
that  stricter  experience  timelines  would  detract  from  the  APDP  and  NCMA  programs, 
based  upon  the  comments  provided,  the  research  team  perceived  that  many  stakeholders 
were  more  concerned  that  the  increased  experience  requirement  would  detract  from  their 
own  career  progression  than  they  were  about  implementing  a  better  system.  The 
research  team  recommends  that  APDP  and  NCMA  both  consider  implementing  stricter 
experience  requirements  for  each  level  of  certification  (i.e.,  2  years,  5  years,  and  10  years 
per  the  ideal  model). 


84 


Fonnal  Education.  It  is  recommended  that  that  the  formal  education  requirements 


for  certain  levels  of  certification  be  increased  imder  APDP  and  the  NCMA  program. 
Under  APDP,  it  is  possible  for  an  individual  to  attain  the  highest  level  of  certification 
while  possessing  only  24  business  related  semester  hours  of  formal  education.  The 
research  team  concludes,  based  upon  the  input  received  fi’om  the  expert  panel,  that  a 
minimum  of  a  Bachelor’s  degree  should  be  required  for  each  level  of  certification.  While 
no  restrictions  of  degree  type  (i.e.,  business,  finance,  etc.)  are  specified,  a  degree  in  itself 
can  be  used  to  assess  an  individual’s  desire  and  ability  to  learn.  NCMA  currently 
requires  a  Bachelor’s  degree  to  attain  the  highest  level  of  certification  (CPCM),  but  only 
requires  one  year  of  college  to  attain  the  first  certification  level  (CACM).  For  the  same 
reasons  stated  above,  the  research  team  recommends  that  the  formal  education 
requirement  for  the  CACM  should  be  heightened  to  require  a  Bachelor’s  degree. 

Procurement  Related  Courses.  The  procurement  related  courses  required  for  each 
level  of  certification  under  the  ideal  model  closely  resemble  the  requirements  for  APDP 
certifications  with  few  exceptions.  The  most  noted  exception  is  that  the  expert  panel 
identified  the  need  for  contract  law  and  contract  negotiation  courses  to  be  required  before 
an  individual  can  attain  Level  One  certification.  Presently  Contract  Law  is  required  for 
APDP  Level  Two,  and  Contract  Negotiation  is  not  presently  required  for  certification  (an 
introduction  to  contract  negotiation  is  presently  included  in  the  DAU  Contract  Pricing 
Course).  Under  NCMA,  only  one  unspecified  acquisition  course  is  required  for  the 
CACM  certification.  For  Level  Two  certification,  the  ideal  model  incorporates  a 
specialty  course  (i.e.,  systems  acquisition,  construction,  value  engineering,  etc.)  in 
addition  to  the  required  APDP  courses.  Presently,  no  such  requirement  exists  under  the 
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APDP  system.  Under  NCMA,  eight  unspecified  acquisition  courses  are  required  for  the 
CPCM  certification.  Finally,  for  Level  Three,  the  ideal  model  includes  such  courses  as 
Advanced  Topics  in  Acquisition,  Management  of  Contracting  Activities,  and  Contract 
Process  Improvements.  Currently,  the  only  courses  required  for  APDP  Level  Three  are 
Executive  Contracting  and  Management  for  Contract  Supervisors.  The  stakeholder 
assessment  supported  the  ideal  model’s  course  requirements  as  improvements  to  both  the 
APDP  and  NCMA  programs.  It  is  therefore  recommended  that  APDP  adopt  these 
changes  to  the  procurement  course  requirements  for  the  three  levels  of  certification.  In 
addition,  it  is  recommended  that  NCMA  adopt  similar  requirements  and  list  specific 
courses  required  for  the  various  levels  of  certification. 

Comprehensive  Examination.  The  ideal  model  includes  a  requirement  for  the 
successful  completion  of  a  comprehensive  examination  at  every  level  of  certification. 
NCMA  currently  requires  comprehensive  examinations  for  each  certification  level,  while 
APDP  does  not.  The  stakeholder  assessment  indicated  a  slight  preference  that  the 
incorporation  of  a  comprehensive  examination  would  be  a  betterment  to  the  existing 
APDP  program.  It  is  therefore  recommended  that  APDP  examine  the  possibility  of 
instituting  such  a  requirement  for  the  demonstration  of  knowledge  in  order  to  enhance  the 
existing  program. 

Re-certification.  The  ideal  model  includes  a  requirement  for  mandatory  re¬ 
certification  every  five  years.  NCMA  currently  incorporates  this  requirement,  while  re¬ 
certification  requirements  are  not  presently  included  in  the  APDP  system.  The 
stakeholder  assessment  indicated  a  strong  preference  that  the  re-certification 
requirements  would  be  an  improvement  to  the  existing  APDP  system.  It  is  therefore 
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recommended  that  APDP  examine  the  possibility  of  instituting  a  five-year  re-certification 
period  to  help  ensure  that  the  contracting  workforce  keeps  current  in  the  contracting 
profession. 

The  ideal  model  also  mandates  an  average  of  40  hours  annually  of  continuing 
contracting  education  in  order  to  be  eligible  for  re-certification.  The  NCMA  re¬ 
certification  system  requires  a  total  of  60  hours  over  the  five-year  re-certification  period, 
with  10  of  those  hours  being  acquired  within  the  last  18  months  of  certification.  Since 
APDP  does  not  have  encompass  re-certification,  there  is  no  formal  requirement  for 
continuing  education.  Obviously,  the  ideal  model  incorporates  more  stringent 
requirements  for  continuing  education  than  NCMA  or  APDP.  The  stakeholder 
assessment  indicated  that  the  more  stringent  requirements  for  continuing  education  would 
improve  both  the  existing  APDP  and  NCMA  programs.  It  is  therefore  recommended 
that  both  programs  incorporate  a  continuing  education  requirement  of  40  hours  per  year. 
This  requirement  will  help  ensure  that  the  contracting  workforce  has  the  opportunity  to 
attend  courses,  hone  skills,  and  keep  current  in  a  changing  environment. 

Suggestions  for  Further  Study 

This  study  accomplished  the  stated  objectives  by  providing  answers  to  the  original 
five  research  questions.  However,  in  formulating  and  presenting  the  ideal  model,  several 
questions  arose  as  to  the  implementation  of  such  a  model.  For  example,  such  questions 
that  remain  to  be  answered  are  what  would  be  the  cost  impact  of  implementing  a 
comprehensive  examination  under  the  APDP  program?  What  would  the  test  be 
comprised  of?  Who  would  be  responsible  for  the  administration  of  the  examinations? 
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Should  previously  certified  individuals  be  required  to  take  the  examination?  How  would 
de-certification  be  implemented  if  a  person  failed  the  examination?  Would  this  have  a 
negative  impact  performance  reports?  These  questions  evolve  from  just  one  element  of 
the  ideal  model.  The  research  team  feels  that  a  follow-on  study  geared  toward  the 
implementation  of  the  ideal  model  would  provide  answers  to  these  questions  as  well  as 
others,  and  identify  the  actual  feasibility  of  implementing  such  a  model. 

Another  suggestion  for  future  research  might  be  to  investigate  the  possibilities  of 
outsourcing  the  APDP  contracting  certification  program.  In  an  era  of  A-76  studies  and 
privatization,  it  may  prove  cost  effective  for  the  Government  to  rely  on  NCMA  to  be  the 
certifying  authority  for  contracting  personnel.  One  major  cost  savings  would  appear  to 
be  alleviating  the  administration  costs  of  the  APDP  system.  In  addition,  since  the  NCMA 
program,  with  minor  modifications,  could  epitomize  the  newly  formulated  ideal  model, 
another  cost  savings  might  be  that  APDP  would  not  have  to  go  through  the  costs  of 
implementing  the  recommended  changes  stated  above.  Furthermore,  if  outsourcing  the 
APDP  contracting  certifications  prove  beneficial,  this  study  could  pave  the  way  for 
outsourcing  other  APDP  certifications  such  as  logistics,  acquisitions,  and/or  program 
management. 

A  final  recommendation  for  future  research  involves  duplicating  the  model 
formulation  conducted  in  this  thesis,  using  a  different  expert  panel  and  comparing  the 
results  with  the  ideal  model  presented  above.  While  the  expert  panel  reached  a  majority 
consensus  on  each  element  of  the  ideal  model,  several  elements  were  decided  by  a  very 
small  margin.  For  example,  the  expert  panel  preferred  three  levels  of  certification  over  a 
two-Ievel  program  54%  to  45%.  If  just  two  panel  members  would  have  voted  differently. 
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the  ideal  model  may  have  had  a  totally  different  outcome.  By  replicating  this  study  using 
a  larger  expert  panel  (i.e.,  30)  and  reaching  a  70%  consensus  on  each  element  of 
certification,  the  ideal  model  presented  herein  can  either  be  supported  or  disputed. 

Summary 

The  Defense  Acquisition  Workforce  Improvement  Act  (DAWIA)  of  1990  requires 
Government  contracting  personnel  to  possess  a  level  of  certification  commensurate  with 
their  grade  and  position  in  order  to  continue  career  progression  in  the  contracting  field. 
The  fundamental  purpose  behind  DAWIA  and  contracting  certifications  is  to  ensure  that 
the  contracting  community  has  a  qualified,  knowledgeable,  and  experienced  personnel 
corps.  Mandatory  certification  requirements  are  presently  governed  by  the  Acquisition 
Professional  Development  Program  (APDP),  as  instituted  by  DAWIA;  however,  the 
National  Contract  Management  Association  (NCMA)  also  sponsors  a  similar,  contracting 
certification  program. 

The  findings  of  this  study  indicate  that  the  ideal  contracting  certification  program, 
according  to  an  expert  panel  of  senior  contracting  officials  and  supported  by  a 
stakeholder  assessment  of  contracting  personnel,  is  actually  a  hybrid  of  the  two  existing 
certification  programs.  The  ideal  contracting  certification  program  does  not  bring  any 
unique  aspects  to  the  certification  process,  but  rather  incorporates  the  best  elements  of 
each  of  the  two  existing  programs.  In  addition,  the  ideal  contracting  certification 
program  encompasses  stricter  requirements  on  several  elements  of  certification  (i.e., 
years  of  experience,  formal  education,  etc.),  and  did  not  relax  a  single  element  from 
either  of  the  existing  programs.  This  indicates  to  the  research  team  that  senior 
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contracting  officials  as  well  as  viable  stakeholders  recognize  the  importance  of  each 
element  of  certification  and  support  the  stricter  requirements  that  will  potentially  lead  to  a 
more  qualified  and  knowledgeable  workforce. 

The  impact  of  this  study  was  aimed  at  ensuring  that  certified  contracting  professionals 
possess  the  levels  of  education,  experience,  and  contracting  knowledge  commensurate 
with  their  certification  level.  By  identifying  potential  deficiencies  in  the  existing 
certification  programs,  recommendations  were  made  to  both  NCMA  and  senior 
government  contracting  personnel  for  potential  improvements  in  both  programs.  The 
formulation  of  the  ideal  contracting  certification  model  will  potentially  lead  to  a  more 
qualified  contracting  field  in  both  the  public  and  private  sector. 
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Appendix  A:  List  of  Expert  Panel  Members 

1 .  Colonel  Steven  Kahne,  OC-ALC/PK  Deputy  Director 

2.  Colonel  David  K.  Hedges,  00-ALC/PK  Deputy  Director 

3.  Colonel  Steven  H.  Sheldon,  ESC/PK  Director 

4.  Colonel  Pamela  R.  Casey,  ASC/PK  Director 

5.  Dr.  Kenneth  Oscar,  SARD  A  ARMY 

6.  Mr.  Elliott  Branch,  HQ  NAVY 

7.  Mr.  John  Webb,  AFIT/LAA,  GS-14 

8.  Mr.  Thomas  Wells,  ESC/PK  Deputy  Director 

9.  Mr.  Morris  Goodrich,  00-ALC/PK  Director 

10.  Mr.  Jerry  C.  Fowler,  AAC/PK  Deputy  Director 

1 1 .  Mr.  Temple  Bowling,  AEDC/PK  Director 

12.  Mr.  Joseph  Farrey,  ASC/PKX,  GS-15 

13.  Mr.  Thomas  C.  Larkin,  AFIT/LSP  (Prior  Army  Contract  Experience),  GS-13 
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Appendix  B:  Initial  Interview  for  Expert  Panel 


1 .  Should  the  ideal  contracting  certification  program  have  multiple  levels  of 

certification?  Why  or  why  not?  If  yes,  how  many  levels  do  you  believe  would  be 
appropriate  and  why? 


2.  How  many  years  of  contracting  experience  should  an  individual  have  to  qualify 
for  certification?  Please  identify  the  years  of  experience  required  with  each  level 
of  certification  identified  in  Question  1 . 


3.  What  levels  of  education  should  an  individual  have  to  qualify  for  certification? 
Please  identify  the  education  level  required  with  each  level  of  certification 
identified  in  Question  1 . 


4.  What  kinds  of  contracting  related  courses  (fundamentals,  law,  negotiation,  etc.) 
should  an  individual  complete  prior  to  being  certified?  Should  different  types  of 
courses  correspond  to  different  levels  of  certification? 


5.  Should  certification  under  the  ideal  contracting  certification  program  require  an 
individual  to  pass  a  comprehensive  examination?  Why  or  why  not? 


6.  Under  an  ideal  contracting  certification  program,  should  periodic  re-certification 
be  required  for  an  individual  to  retain  his/her  certification?  Why  or  why  not?  If 
you  believe  re-certification  should  be  required,  how  often  should  this  be 
accomplished  and  what  should  it  entail? 
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Appendix  C:  Delphi  Questionnaire 


1 .  How  many  levels  of  certification  do  you  believe  would  be  appropriate  for  the  ideal 
contracting  certification  program? 

_  Two  Levels  (30%) 

_  Three  Levels  (40%) 

2.  How  many  years  of  contracting  experience  should  an  individual  have  to  possess  in 
order  to  qualify  for  certification  under  a  TWO-LEVEL  program? 

LEVEL  ONE: 

_  Three  Years  (33.3%) 

_ -  Four  Years  (33.3%) 

_  Five  Years  (33.3%) 

LEVEL  TWO: 

_  Seven  Years  (33.3%) 

_  Ten  Years  (66.7%) 

3.  How  many  years  of  contracting  experience  should  an  individual  have  to  possess  in 
order  to  qualify  for  certification  under  a  THREE-LEVEL  program? 

LEVEL  ONE: 

_  No  Experience  (25%) 

_  Two  Years  (25%) 

_  Three  Years  (50%) 
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LEVEL  TWO: 


_  Three  Years  (25%) 

_  Four  Years  (25%) 

_  Five  Years  (25%) 

_  Six  Years  (25%) 

LEVEL  THREE: 

_  Five  Years  (25%) 

_  Eight  Years  (25%) 

_  Ten  Years  (50%) 

4.  What  level  of  education  should  an  individual  have  to  possess  in  order  to  qualify  for 
certification  under  a  TWO-LEVEL  program? 

LEVEL  ONE: 

_  B.S./B.A.  (100%) 

_  Other  -  Please  Specify 

LEVEL  TWO: 

_  Masters  Degree  (66.7%) 

_____  Additional  Professional  Certification  (33.3%) 

_  Other  -  Please  Specify 
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5.  What  level  of  education  should  an  individual  have  to  possess  in  order  to  qualify  for 
certification  under  a  THREE-LEVEL  program? 

LEVEL  ONE: 

_  B.S./B.A.  (50%) 

_  Education  levels  are  not  relevant  (50%) 

LEVEL  TWO: 

_  B.S./B.A.  (50%) 

_  Education  levels  are  not  relevant  (50%) 

LEVEL  THREE: 

_  B.S./B.A.  (50%) 

_  Education  levels  are  not  relevant  (50%) 

6.  Please  indicate  to  which  level  of  certification  the  following  contracting  courses 
should  be  required  under  a  TWO-LEVEL  program. 

Contracting  Fundamentals: 

_ Level  One  _ Level  Two  _ Not  Relevant 

Contract  Pricing; 

_ Level  One  _ Level  Two  _ Not  Relevant 

Specialty  Course  (Depending  on  Assignment): 

_ Level  One  _ Level  Two  _ Not  Relevant 

Intermediate  Contracting: 

Level  One  _ Level  Two  _ Not  Relevant 


95 


Intermediate  Pricing: 

_ Level  One  _ Level  Two  Not  Relevant 

Contract  Law: 

_ Level  One  _ Level  Two  Not  Relevant 

Contract  Negotiation: 

_ _ Level  One  _ Level  Two  Not  Relevant 

Executive  Contracting: 

_ Level  One  _ Level  Two  Not  Relevant 

Advanced  Topics  in  Acquisition: 

_  Level  One  _  Level  Two  Not  Relevant 

Management  of  Contracting  Activities: 

_ Level  One  _ Level  Two  Not  Relevant 

Contract  Process  Improvements: 

_  Level  One  ■  Level  Two  _ Not  Relevant 

7.  Please  indicate  to  which  level  of  certification  the  following  contracting  courses 
should  be  required  under  a  THREE-LEVEL  program. 

Contracting  Fundamentals: 

_ Level  1  _ Level  2  _ Level  3  _  N/R 
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Contract  Pricing: 

_ Level  1  _ Level  2  _ Level  3  _ N/R 

Specialty  Course  (Depending  on  Assignment): 

_ Level  1  _ Level  2  _ Level  3  _ N/R 

Intermediate  Contracting: 

_ Level  1  _ Level  2  _ Level  3  _ N/R 

Intermediate  Pricing: 

.  Level  1  _ Level  2  _ Level  3  _ N/R 

Contract  Law: 

_ Level  1  _ Level  2  _ Level  3  _ N/R 

Contract  Negotiation: 

_ Level  1  _ _ Level  2  _ Level  3  _ N/R 

Executive  Contracting: 

_ Level  1  _ Level  2  _ Level  3  _ N/R 

Advanced  Topics  in  Acquisition: 

_ Level  1  _ Level  2  _ Level  3  _ N/R 

Management  of  Contracting  Activities: 

Level  1  _  Level  2  _ Level  3  _ N/R 
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Contract  Process  Improvements: 

_ Level  1  _ _  Level  2  _ Level  3  _ N/R 

8.  Should  certification  under  the  ideal  contracting  certification  program  require  an 
individual  to  pass  a  comprehensive  examination? 

_ Yes,  for  ALL  levels  of  certification  (60%) 

_  Yes,  but  only  for  the  most  advanced  certification  level  (10%) 

_ No,  an  examination  would  not  be  beneficial  (30%) 

9.  Under  an  ideal  contracting  certification  program,  should  periodic  RE¬ 
CERTIFICATION  be  required  for  an  individual  to  retain  his/her  certification? 

_ _  Yes,  via  a  periodic  examination  (10%) 

_  Yes,  via  experience  and  continuing  education  (50%) 

_  Yes,  via  experience,  continuing  education,  and  an  examination  (10%) 

_ No  (30%) 

10.  Assuming  a  contracting  certification  program  with  mandatory  re-certification, 
how  long  should  an  individual’s  certification  remain  valid? 

_  1  year 

_  2  years 

_  3  years 

_  4  years 

_  5  years 
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1 1 .  Assuming  a  contracting  certification  program  with  mandatory  re-certification,  how 
much  continuing  education  (on  average)  should  be  required  to  maintain  an  individual’s 
certification? 


30  hours  per  year  (10%) 
40  hours  per  year  (20%) 
Other  -  please  specify 
_  hours  per  year 
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Appendix  D:  Stakeholder  Assessment  Questionnaire 


Please  use  the  following  table  to  answer  questions  1-8. 


Requirements 

Ideal  Model 

APDP 

NCMA 

Experience 

Level  1 

2  years 

1  year 

1  year 

Level  2 

5  years 

2  years 

2  years 

Level  3 

10  years 

4  years 

N/A 

1.  How  do  the  Level  1  experience  requirements  of  the  proposed  program  measure  up  to 
those  of  the  APDP  program? 


Much  Worse  Worse  Similar  Better  Much  Better 

2.  How  do  the  Level  1  experience  requirements  of  the  proposed  program  measure  up  to 
those  of  the  NCMA  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

3.  Is  this  difference  important?  Why? 


4.  How  do  the  Level  2  experience  requirements  of  the  proposed  program  measure  up  to 
those  of  the  APDP  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

5.  How  do  the  Level  2  experience  requirements  of  the  proposed  program  measure  up  to 
those  of  the  NCMA  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

6.  Is  this  difference  important?  Why? 

7.  How  do  the  Level  3  experience  requirements  of  the  proposed  program  measure  up  to 
those  of  the  APDP  program? 

Much  Worse  Worse  Similar  Better  Much  Better 
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8.  Is  this  difference  important?  Why? 


Please  use  the  following  table  to  answer  Questions  9-16. 


Requirements 

Ideal  Model 

APDP 

NCMA 

Formal  Education 

Level  1 

Bachelor's  Degree 

Bachelors  Degree  or 
24  business  related 
semester  hours,  or  10 
years  experience 

1  year  college 

Level  2 

same  as  above 

same  as  above 

Bachelor’s  Degree 

Level  3 

same  as  above 

same  as  above 

N/A 

9.  How  do  the  Level  1  education  requirements  of  the  proposed  program  measure  up  to 
those  of  the  APDP  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

10.  How  do  the  Level  1  education  requirements  of  the  proposed  program  measure  up  to 
those  of  the  NCMA  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

1 1 .  Is  this  difference  important?  Why? 


12.  How  do  the  Level  2  education  requirements  of  the  proposed  program  measure  up  to 
those  of  the  APDP  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

13.  How  do  the  Level  2  education  requirements  of  the  proposed  program  measure  up  to 
those  of  the  NCMA  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

14.  Is  this  difference  important?  Why? 
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15.  How  do  the  Level  3  education  requirements  of  the  proposed  program  measure  up  to 
those  of  the  APDP  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

16.  Is  this  difference  important?  Why? 


Please  use  the  following  table  to  answer  Questions  17-24. 


Requirements 

Ideal  Model 

APDP 

NCMA 

Procurement  Courses 

Level  1 

Contracting 
Fundamentals, 
Contract  Pricing, 
Contract  Law, 

Contract  Negotiation 

Contracting 
Fundamentals, 
Contract  Pricing 

One  acquisition  course 

Level  2 

Intermediate 
Contracting, 
Intermediate  Pricing, 
Specialty  Course 

Intermediate 
Contracting, 
Intermediate  Pricing, 
Contract  Law 

Eight  procurement-related 
courses 

Level  3 

Executive 

Contracting, 

Advanced  Topics  in 
Acquisition, 
Management  of 
Contracting  Activities, 
Contract  Process 
Improvements 

Executive 
Contracting, 
Management  for 
Contract  Supervisors 

N/A 

17.  How  do  the  Level  1  course  requirements  of  the  proposed  program  measure  up 
to  those  of  the  APDP  program? 


Much  Worse  Worse  Similar  Better  Much  Better 

1 8.  How  do  the  Level  1  course  requirements  of  the  proposed  progrm  measure  up 
to  those  of  the  NCMA  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

1 9.  Is  this  difference  important?  Why? 
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20.  How  do  the  Level  2  course  requirements  of  the  proposed  program  measure  up 
to  those  of  the  APDP  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

21 .  How  do  the  Level  2  course  requirements  of  the  proposed  program  measure  up 
to  those  of  the  NCMA  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

22.  Is  this  difference  important?  Why? 


23.  How  do  the  Level  3  course  requirements  of  the  proposed  program  measure  up 
to  those  of  the  APDP  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

24.  Is  this  difference  important?  Why? 


Please  use  the  following  table  to  answer  Question  25-26. 


Ideal  Certification 
Model 

APDP 

NCMA 

Comprehensive 

Examination 

Yes,  all  levels 

No 

Yes,  both  levels 

25.  Is  the  requirement  to  pass  a  comprehensive  examination  an  enhancement  relative  to 
the  existing  APDP  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

26.  Is  this  difference  important?  Why? 


Please  use  the  following  table  to  answer  Questions  27-28. 


Ideal  Model 

APDP 

NCMA 

Re-certification 

Yes,  via  experience 
and  continuing 
education 

No 

Yes,  via  experience 
and  continuing 
education 
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27.  Is  the  proposed  re-certification  requirement  an  enhancement  relative  to  the 
existing  APDP  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

28.  Is  this  difference  important?  Why? 


Please  use  the  following  table  to  answer  Question  29-3 1 . 


Ideal  Model 

APDP 

NCMA 

Duration  of  Certification 

5  years 

Indefinite 

5  years 

29.  How  does  the  proposed  duration  of  certification  measure  up  to  that  of  the  existing 
APDP  program? 


Much  Worse  Worse  Similar  Better  Much  Better 

30.  How  does  the  proposed  duration  of  certification  measure  up  to  that  of  the  existing 
NCMA  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

3 1 .  Is  this  difference  important?  Why? 


Please  use  the  following  table  to  answer  Question  32-34. 


Ideal  Model 

APDP 

NCMA 

Continuing  Education 

40  hours  per  year 

None 

60  hours  over  a  5-year 
span  with  10  of  those 

60  completed  within 
the  last  1 8  months. 

32.  How  do  the  proposed  continuing  education  requirements  measure  up  to  that  of  the 
existing  APDP  program? 


Much  Worse  Worse  Similar  Better  Much  Better 

33.  How  do  the  proposed  continuing  education  requirements  measure  up  to  that  of  the  • 

existing  NCMA  program? 

Much  Worse  Worse  Similar  Better  Much  Better 

34.  Is  this  difference  important?  Why? 
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